Intelligent monitoring system and method for nursing baby

Abstract

The invention discloses an intelligent monitoring system and method for nursing infants, and the method comprises the following steps: S1, collecting continuous frames of infrared images, carrying out the preprocessing, and extracting feature vectors; s2, constructing a classification model for identifying infant micro-motion types, and training the classification model to ensure accurate identification of the classification model; s3, acquiring a real-time infrared monitoring video image of the infant, performing similarity analysis on continuous frames, judging the range of the continuous frames of the micro-action, and extracting continuous frame images; and S4, feature vectors are extracted from the extracted continuous frame images, the type of the infant micro-motion is identified, and user side reminding and analysis are carried out. Aiming at low-light environment and micro-motion detection in baby nursing, through combination of infrared imaging and optical flow feature extraction and fusion of a perceptual hash array and a Hamming distance, real-time and reliable baby state monitoring service is provided for parents or caregivers, and the safety and convenience of baby nursing are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image communication, and in particular to an intelligent monitoring system and method for caring for infants. Background Art

[0002] In modern society, with the changes in family structure and the accelerated pace of work, more and more parents need to take care of their babies in addition to their busy work. However, due to various reasons, such as work and housework, parents are often unable to accompany their babies all the time, especially when the baby is sleeping. Babies may have various micro-movements during the process of falling asleep, such as turning over, kicking legs, waving hands, etc. These micro-movements often reflect the baby's sleeping state, health status and possible needs.

[0003] Traditional infant care methods mainly rely on manual observation, that is, parents or caregivers perceive the baby's condition through vision and hearing. However, this method has many shortcomings. First, manual observation requires parents or caregivers to remain highly vigilant at all times, which is undoubtedly a huge mental burden for parents who take care of the baby for a long time. Second, manual observation is often interfered by multiple factors such as the environment, light, and sound, which can easily lead to misjudgment or omission of important information.

[0004] Therefore, real-time monitoring and analysis of infants' micro-movements when they fall asleep is of great significance to ensure their healthy growth. Summary of the invention

[0005] The present invention overcomes the deficiencies of the prior art and provides an intelligent monitoring system and method for caring for infants.

[0006] To achieve the above object, the technical solution adopted by the present invention is: an intelligent monitoring method for caring for babies, comprising the following steps:

[0007] S1, collecting continuous frame infrared images of infants under different micro-movements, preprocessing the continuous frame infrared images, and extracting feature vectors of the continuous frame infrared images;

[0008] S2. Based on the feature vector, a classification model for identifying the types of infant micro-movements is constructed. A large number of labeled micro-movement data sets are used to train the classification model. The parameters of the classification model are adjusted using an optimization algorithm to ensure accurate recognition of the classification model.

[0009] S3, obtaining a real-time infrared monitoring video image of the baby, performing grayscale processing on the video image, performing similarity analysis on continuous frames in the video image, determining the range of continuous frames of micro-motions according to the similarity analysis, and extracting continuous frame images with micro-motions;

[0010] S4. Extract feature vectors from the extracted continuous frame images and input them into a classification model to identify the types of infant micro-movements. The identified micro-movement types are reminded and analyzed on the user side to realize intelligent monitoring of infant care.

[0011] In a preferred embodiment of the present invention, the step S1 includes the following sub-steps:

[0012] S11, collect continuous frame infrared images of infants under different micro-movements;

[0013] S12, converting the collected continuous frame infrared images into grayscale images, and performing noise reduction processing to ensure image quality;

[0014] S13, for each continuous frame interval of the micro-motion, calculating the horizontal and vertical optical flows of each frame of the infrared video, and obtaining the motion vector of each pixel between the continuous frames; including the motion direction and speed involved;

[0015] S14, dividing the direction of the optical flow vector into several direction intervals, counting the number of optical flow vectors in each direction interval to form an optical flow histogram, and extracting the amplitude characteristics of the optical flow, counting the distribution of the optical flow amplitude, and obtaining information about the motion intensity;

[0016] S15, perform multi-scale transformation on the infrared image, calculate the optical flow features at each scale, capture the motion information at different scales, and fuse the optical flow features at different scales to form a comprehensive optical flow feature representation;

[0017] S16. Perform edge detection on the infrared image using an edge detection algorithm to obtain an edge map, combine the edge information of the edge map with the motion information of the optical flow feature map, enhance the optical flow features of the edge area, and fuse the optical flow features with the grayscale features and edge features of the infrared image to obtain a comprehensive feature vector.

[0018] In a preferred embodiment of the present invention, in the step S13, calculating the horizontal and vertical optical flows of each frame of infrared video includes the following steps:

[0019] S131, assuming that the image brightness is constant, let I(x, y, t) be the brightness value of the image at position (x, y) and time t, I(x, y, t) = I(x + Δx, y + Δy, t + Δt);

[0020] S132. Perform Taylor expansion on the brightness value of the image at the position and time, ignoring the higher-order terms: I(x,y,t)≈I(x+Δx,y+Δy,t+Δt)+I x Δx+I y Δy+I t Δt; where I x ,I yand I t are the partial derivatives of the image in the x, y and t directions respectively; due to the constant brightness, it is simplified to: I x Δx+I y Δy+I t Δt=0;

[0021] S133, assuming Δx=u and Δy=v, where u is the horizontal speed of each pixel and v is the vertical speed of each pixel, then I x u+I y v+I t Δt=0; Matrix form:

[0022] S134. Within the local window, the motion is uniform, and the matrix form is: Where n is the number of pixels in the local window; by solving the least squares method, we get: Where A is the coefficient matrix, b is a constant term vector, A T is the transpose of matrix A;

[0023] S135 , repeat the above steps for each frame image in the continuous frame interval of each micro-motion, and calculate the horizontal and vertical optical flows of each frame image.

[0024] In a preferred embodiment of the present invention, the step S14 specifically includes the following steps:

[0025] S141, the direction of the optical flow vector is divided into 8 intervals, each interval corresponds to an angle range of 45 degrees;

[0026] S142, for each pixel's optical flow vector Calculate direction θ and magnitude Where, direction θ = arctan2(v,u), arctan2 is the four-quadrant inverse tangent function, and the return value range is [-180,180] degrees;

[0027] S143, mapping the direction θ to a range of 0 to 360 degrees, determining which direction interval it belongs to, and counting the number of optical flow vectors in each direction interval to form an optical flow histogram;

[0028] S144. Count the optical flow amplitude values ​​of all pixels to obtain information about motion intensity.

[0029] In a preferred embodiment of the present invention, the step S15 specifically includes the following steps:

[0030] S151, using a Gaussian filter to perform blur processing on the infrared image, removing high-frequency details in the image, and performing a downsampling operation on the blurred image;

[0031] S152, taking the reduced image as input, repeatedly performing Gaussian blur and downsampling steps to construct the next layer of the Gaussian pyramid until the required Gaussian pyramid level is reached;

[0032] S153, taking each level image of the Gaussian pyramid as input, for each scale image I s , calculate the optical flow features between adjacent frames and obtain the optical flow histogram H s , capturing motion information at different scales;

[0033] S154, the optical flow histogram H at each scale s Assign a weight w s , the optical flow histogram H at different scales s Perform weighted summation to obtain the comprehensive optical flow histogram H, H = ∑w s H s .

[0034] In a preferred embodiment of the present invention, the step S16 specifically includes the following steps:

[0035] S161. Calculate the gradient of the image in the x and y directions:

[0036] Calculate the gradient magnitude M and direction θ: θ=arctan2(G y ,G x );

[0037] S162, for each pixel, check the neighboring pixels in the gradient direction, and if the gradient magnitude of the current pixel is not a local maximum, set its gradient magnitude to 0;

[0038] S163, using two thresholds T high and T low Perform threshold processing on the gradient amplitude; where T high For detecting strong edges, T low Used to detect weak edges; when the gradient amplitude is greater than T high , then the pixel is considered to be a strong edge; when the degree amplitude is T low and T high When the gradient amplitude is less than T low , then the pixel is suppressed;

[0039] S164, checking the neighboring pixels of the weak edge pixel, if there is a strong edge pixel among the neighboring pixels, retaining the weak edge pixel to obtain an edge map;

[0040] S165. For the optical flow features u and v of the edge area, weighted enhancement is adopted, that is, for each pixel, when the pixel is an edge pixel in the edge map E, the optical flow features u(x, y) and v(x, y) of the pixel are multiplied by a weight w;

[0041] S166, the grayscale feature map I, the edge feature map E, the optical flow feature maps u and v are fused to form a comprehensive feature vector,

[0042] In a preferred embodiment of the present invention, the step S2 includes the following sub-steps:

[0043] S21, using the extracted feature vector as input feature and the micro-movement type as output target, constructing a classification model capable of identifying the type of infant micro-movement;

[0044] The identification of the infant micro-movement type adopts a machine learning algorithm, and the micro-movement type identification: Where e is the base of natural logarithms; x 1 ...x n is the eigenvector X, β 0 ...β n is the model parameter β, P is the type of micro-motion identified;

[0045] S22, labeling a large amount of infant micro-movement video data, clarifying the specific movement type of the infant in each video, and pre-processing the labeled data; including grayscale conversion and noise reduction processing involved;

[0046] S23, dividing the processed data into a training set and a validation set, using the training set data to train the classification model, using the validation set data to validate the classification model, using an optimization algorithm to adjust the parameters of the classification model, and through multiple iterations of training, enabling the classification model to accurately identify different types of micro-motions;

[0047] In the classification model training, the error between the predicted value and the true value is quantified: Among them, Loss is the cross entropy loss function; y i is the true label of the i-th sample; σ(z i ) is the predicted probability of the i-th sample; N is the number of samples;

[0048] Use an optimization algorithm to minimize the loss function, adjust the parameters of the model, and calculate the gradient of the loss function with respect to each parameter: Among them, w is the weight;

[0049] Where b is the bias;

[0050] Update the parameters using the gradient descent method: Among them, α is the learning rate, which controls the step size of parameter update.

[0051] In a preferred embodiment of the present invention, the step S3 includes the following sub-steps:

[0052] S31, acquiring infrared monitoring video images of the baby in real time, and converting each frame of the video image into a grayscale image;

[0053] S32, reducing each frame of grayscale image to a fixed size, calculating the grayscale average value of all pixels in the reduced image, and comparing the grayscale value of each pixel with the average value to obtain a hash value of each frame of image;

[0054] Calculate the grayscale average of the pixel: Where, μ is the grayscale average of all pixels in the reduced image; C and D are the number of rows and columns of the reduced image, respectively; I(i, j) is the grayscale value of the reduced image at position (i, j);

[0055] Compare the grayscale value of each pixel of the image with the average value μ, and let the generated binary string be B, where B k is the kth bit of the string (k = 0, 1, ..., 63). For each pixel I (i, j), calculate its corresponding binary value B k : Where k is the index of the pixel in the 64-bit string;

[0056] Convert the generated 64-bit binary string B to a decimal hash value:

[0057]

[0058] S33, calculating the Hamming distance between the hash values ​​of two consecutive frames of images, setting the Hamming distance threshold, and judging the similarity of the two frames of images and whether there is micro-motion according to the size of the Hamming distance and the threshold;

[0059] The Hamming distance between the hash values ​​of the two frames of images: Where HD is the Hamming distance between the hash values ​​of two consecutive frames; L is the length of the hash value, which is 64 bits; U 1 (i) and U 2 (i) are the i-th digits of the two hash values;

[0060] Judging similarity and micro-movements: Among them, S is the judgment result; T is the Hamming distance threshold; 0 means the baby has no micro-movement; 1 means the baby has micro-movement;

[0061] S34, randomly selecting a frame from the video sequence as the starting frame, and then calculating the Hamming distance between the subsequent frames and the starting frame frame by frame, when the Hamming distance between a certain frame image and the starting frame is less than a threshold and the number of frames between them is greater than half of the video period, the frame is used as the ending frame, and the frame sequence between the starting frame and the ending frame is a key frame sequence;

[0062] The starting frame F start With the termination frame F end Between, key frame sequence extraction: HD i <Tand Where i is the index of the current frame; start is the index of the starting frame; J is the video period; the starting frame F start With the termination frame F end The frame sequence between is a key frame sequence;

[0063] S35, extracting the key frame sequence determined to have micro-motions for micro-motion recognition.

[0064] In a preferred embodiment of the present invention, the step S4 includes the following sub-steps:

[0065] S41, extracting optical flow features from the extracted continuous frame images, including the optical flow calculation, optical flow histogram and optical flow amplitude features involved;

[0066] S42, enhancing the edge blurred image of the continuous frame image by using the extracted optical flow features, and fusing the optical flow features with the grayscale features and edge features of the infrared image to obtain a comprehensive feature vector;

[0067] S43, inputting the extracted feature vector into a classification model to identify the type of micro-motion;

[0068] S44, the identified micro-movement type is reminded through the user terminal device, and the identified micro-movement type and time are recorded, and an analysis report of the baby's behavior pattern is provided to the user to help the user better understand the baby's behavior habits and health status.

[0069] The present invention provides an intelligent monitoring system for an intelligent monitoring method for caring for infants, comprising:

[0070] An image collection module, used to collect continuous frame infrared images under different micro-motions, as well as real-time infrared monitoring videos of infants;

[0071] Image processing module, used for image preprocessing and grayscale processing;

[0072] A feature extraction module is used to extract feature vectors of continuous frame infrared images;

[0073] A classification model is used to identify the type of infant micro-movements based on the extracted feature vectors;

[0074] A similarity analysis module is used to perform similarity analysis of consecutive frames on the acquired real-time infrared monitoring video images of infants;

[0075] A continuous frame image extraction module is used to determine the range of continuous frames of micro-motions based on similarity analysis, and to extract continuous frame images with micro-motions;

[0076] The reminder analysis module is used to remind and analyze the identified micro-action types on the user side.

[0077] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0078] (1) The present invention provides an intelligent monitoring system and method for baby care. Aiming at low-light environment and micro-motion detection in baby care, the system effectively overcomes the problems of blurred edges and low resolution of infrared images by combining infrared imaging with optical flow feature extraction, so that the system can accurately identify the tiny movements of babies in low-light conditions. It also efficiently extracts key frames by fusing perceptual hash arrays and Hamming distance, providing a high-quality data basis for action classification. Multi-scale optical flow analysis and edge enhancement technology are then used to further improve the accuracy of micro-motion recognition, providing parents or caregivers with real-time and reliable baby status monitoring services, thereby enhancing the safety and convenience of baby care.

[0079] (2) In the present invention, by calculating the horizontal and vertical optical flow features of each frame of infrared video, the contour information is supplemented, the micro-movements of the infant are effectively extracted, the problems of edge blur and low resolution in infrared imaging are overcome, and the clarity of image details is improved, so that the tiny movements of the infant can be accurately captured, providing a basis for subsequent action recognition.

[0080] (3) In the present invention, each frame image in the video is converted into a grayscale image, reduced to a fixed size, the grayscale average is calculated, a hash value is generated, and the similarity between two frames of images is quickly determined by the Hamming distance, thereby reducing the amount of calculation, improving the processing speed, and effectively extracting a key frame sequence containing complete action information, reducing the amount of data, and improving the efficiency of subsequent processing.

[0081] (4) In the present invention, by fusing the optical flow features with the grayscale features, edge features, etc. of the infrared image, a more comprehensive feature vector is formed to improve the accuracy of small motion recognition, reduce the noise impact of a single feature, and enhance the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0083] Figure 1 is a flow chart of an intelligent monitoring method for baby care according to a preferred embodiment of the present invention;

[0084] Figure 2 It is a structural diagram of an intelligent monitoring system for caring for babies according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0085] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0086] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0087] Application Overview:

[0088] The principle of infrared imaging is mainly based on the infrared radiation (thermal radiation) emitted by objects, which is converted into visible images through infrared sensors. All objects emit electromagnetic waves according to their temperature. This phenomenon is called thermal radiation or infrared radiation. The higher the temperature of the object, the stronger the infrared radiation emitted. The sensor in the infrared camera can detect and capture this infrared radiation and convert it into electrical signals. After amplification and processing, the electrical signals received by the sensor form an image representing the temperature distribution on the surface of the object.

[0089] Given that babies often sleep in low-light environments, infrared video surveillance is often used. Since infrared imaging is based on temperature distribution rather than color and texture information, the edges of the target may not be as sharp as visible light images, which may cause blurred edges. In addition, compared with visible light cameras, the resolution of infrared cameras is usually lower, resulting in unclear image details and making it difficult to distinguish the baby's micro-movements.

[0090] In response to the above technical problems, the concept of the present invention is to propose an intelligent monitoring system and method for caring for infants, which captures the detailed changes of the infant's micro-movements, supplements the image edge blur, performs feature fusion and micro-movement recognition, and accurately classifies the types of tiny movements, thereby realizing intelligent monitoring with user-side reminders.

[0091] like Figure 1 As shown, an intelligent monitoring method for baby care includes the following steps:

[0092] S1, collecting continuous frame infrared images of infants under different micro-movements, preprocessing the continuous frame infrared images, and extracting feature vectors of the continuous frame infrared images;

[0093] S2. Based on the feature vector, a classification model for identifying the types of infant micro-movements is constructed. A large number of labeled micro-movement data sets are used to train the classification model. The parameters of the classification model are adjusted using an optimization algorithm to ensure accurate recognition of the classification model.

[0094] S3, obtaining a real-time infrared monitoring video image of the baby, performing grayscale processing on the video image, performing similarity analysis on continuous frames in the video image, determining the range of continuous frames of micro-motions according to the similarity analysis, and extracting continuous frame images with micro-motions;

[0095] S4. Extract feature vectors from the extracted continuous frame images and input them into a classification model to identify the types of infant micro-movements. The identified micro-movement types are reminded and analyzed on the user side to realize intelligent monitoring of infant care.

[0096] In some specific embodiments, the step of S1 includes the following sub-steps:

[0097] S11, collect continuous frame infrared images of infants under different micro-movements;

[0098] S12, converting the collected continuous frame infrared images into grayscale images, and performing noise reduction processing to ensure image quality;

[0099] S13, for each continuous frame interval of the micro-motion, calculating the horizontal and vertical optical flows of each frame of the infrared video, and obtaining the motion vector of each pixel between the continuous frames; including the motion direction and speed involved;

[0100] S14, dividing the direction of the optical flow vector into several direction intervals, counting the number of optical flow vectors in each direction interval to form an optical flow histogram, and extracting the amplitude characteristics of the optical flow, counting the distribution of the optical flow amplitude, and obtaining information about the motion intensity;

[0101] S15, perform multi-scale transformation on the infrared image, calculate the optical flow features at each scale, capture the motion information at different scales, and fuse the optical flow features at different scales to form a comprehensive optical flow feature representation;

[0102] S16. Perform edge detection on the infrared image using an edge detection algorithm to obtain an edge map, combine the edge information of the edge map with the motion information of the optical flow feature map, enhance the optical flow features of the edge area, and fuse the optical flow features with the grayscale features and edge features of the infrared image to obtain a comprehensive feature vector.

[0103] It should be noted that by calculating the horizontal and vertical optical flow features of each frame of infrared video, the contour information is supplemented, the baby's micro-movements are effectively extracted, the problems of edge blur and low resolution in infrared imaging are overcome, and the clarity of image details is improved, so that the baby's tiny movements can be accurately captured, providing a basis for subsequent action recognition.

[0104] In this embodiment, in step S11, continuous frames of infrared images are collected by an infrared camera in a low-light environment, and micro-movements include but are not limited to turning over, grasping, and kicking.

[0105] In this embodiment, in step S12, the noise reduction processing is performed through one of Gaussian filtering, mean filtering or median filtering, such as Gaussian filtering. The kernel size (such as 3x3, 5x5, etc.) and standard deviation (to control the degree of smoothness) of the filter can be set, and then the filter is applied to the image to check whether the noise is reduced and whether important information (such as edges, details, etc.) is retained to ensure image quality.

[0106] In this embodiment, in step S13, the horizontal and vertical optical flows of each frame of infrared video are calculated by Lucas-Kanade optical flow algorithm, including the following steps:

[0107] S131. Assume that the image brightness is constant, that is, the image brightness does not change within a small time interval; let I(x,y,t) be the brightness value of the image at position (x,y) and time t, I(x,y,t)=I(x+Δx,y+Δy,t+Δt);

[0108] S132. Perform Taylor expansion on the brightness value of the image at the position and time, ignoring the higher-order terms: I(x,y,t)≈I(x+Δx,y+Δy,t+Δt)+I x Δx+I y Δy+I t Δt; where I x ,I y and I t are the partial derivatives of the image in the x, y and t directions respectively; due to the constant brightness, it is simplified to: Ix Δx+I y Δy+I t Δt=0;

[0109] S133, assuming Δx=u and Δy=v, where u is the horizontal speed of each pixel and v is the vertical speed of each pixel, then I x u+I y v+I t Δt=0; Matrix form:

[0110] S134. Within the local window, the motion is uniform, and the matrix form is: Where n is the number of pixels in the local window; by solving the least squares method, we get: Where A is the coefficient matrix, b is a constant term vector, A T is the transpose of matrix A. The transpose operation of a matrix is ​​to swap the rows and columns of the matrix, that is, the element in the i-th row and j-th column of the original matrix becomes the element in the j-th row and i-th column of the transposed matrix;

[0111] S135 , repeat the above steps for each frame image in the continuous frame interval of each micro-motion, and calculate the horizontal and vertical optical flows of each frame image.

[0112] In this embodiment, the step S14 specifically includes the following steps:

[0113] S141, the direction of the optical flow vector is divided into 8 intervals, each interval corresponds to an angle range of 45 degrees (from 0 to 360 degrees);

[0114] S142, for each pixel's optical flow vector Calculate direction θ and magnitude Where, direction θ = arctan2(v,u), arctan2 is the four-quadrant inverse tangent function, and the return value range is [-180,180] degrees;

[0115] S143, mapping the direction θ to a range of 0 to 360 degrees, determining which direction interval it belongs to, and counting the number of optical flow vectors in each direction interval to form an optical flow histogram;

[0116] S144. Count the optical flow amplitude values ​​of all pixels to obtain information about motion intensity.

[0117] In this embodiment, the step S15 specifically includes the following steps:

[0118] S151, blurring the infrared image using a Gaussian filter to remove high-frequency details in the image, and downsampling the blurred image (i.e., reducing the image size by half); wherein the downsampling operation generates a new image of a smaller size by selecting sampling or interpolation techniques (such as average value, nearest neighbor, etc.) at regular intervals of pixels;

[0119] S152, taking the reduced image as input, repeatedly performing Gaussian blurring and downsampling steps, and constructing the next layer of the Gaussian pyramid, until the required Gaussian pyramid level is reached; wherein each layer is a blurred and downsampled version of the previous layer, and the downsampled images of each level are used as a layer of the pyramid, and are arranged in sequence to form a pyramid structure;

[0120] S153, taking each level image of the Gaussian pyramid as input, for each scale image I s , calculate the optical flow features between adjacent frames and obtain the optical flow histogram H s , capturing motion information at different scales; the calculation steps of optical flow features are consistent with those in S13;

[0121] S154, the optical flow histogram H at each scale s Assign a weight w s , the optical flow histogram H at different scales s Perform weighted summation to obtain the comprehensive optical flow histogram H, H = ∑w s H s ; where the weights are determined based on the importance of the scale, and usually the sum of the weights is 1, i.e. Σw s =1.

[0122] In this embodiment, the step S16 specifically includes the following steps:

[0123] S161. Use the Sobel operator to calculate the gradient of the image in the x and y directions: Calculate the gradient magnitude M and direction θ: θ=arctan2(G y ,G x );

[0124] S162, for each pixel, check the neighboring pixels in the gradient direction, and if the gradient magnitude of the current pixel is not a local maximum, set its gradient magnitude to 0;

[0125] S163, using two thresholds T high and T low Perform threshold processing on the gradient amplitude; where T high For detecting strong edges, T lowUsed to detect weak edges; when the gradient amplitude is greater than T high , then the pixel is considered to be a strong edge; when the degree amplitude is T low and T high When the gradient amplitude is less than T low , then the pixel is suppressed;

[0126] S164, checking the neighboring pixels of the weak edge pixel, if there is a strong edge pixel among the neighboring pixels, retaining the weak edge pixel to obtain an edge map;

[0127] S165. For the optical flow features u and v of the edge area, weighted enhancement is adopted, that is, for each pixel, when the pixel is an edge pixel in the edge map E (that is, E(x, y) = 1), the optical flow features u(x, y) and v(x, y) of the pixel are multiplied by the weight w;

[0128] S166, the grayscale feature map I, the edge feature map E, the optical flow feature maps u and v are fused to form a comprehensive feature vector,

[0129] In some specific embodiments, the step of S2 includes the following sub-steps:

[0130] S21, using the extracted feature vector as input feature and the micro-movement type as output target, constructing a classification model capable of identifying the type of infant micro-movement;

[0131] S22, labeling a large amount of infant micro-movement video data, clarifying the specific movement type of the infant in each video, and pre-processing the labeled data; including grayscale conversion and noise reduction processing involved;

[0132] S23. Divide the processed data into a training set and a validation set, use the training set data to train the classification model, use the validation set data to validate the classification model, use an optimization algorithm to adjust the parameters of the classification model, and through multiple iterative training, enable the classification model to accurately identify different types of micro-motions.

[0133] In this embodiment, in step S21, the identification of infant micro-movement types uses a machine learning algorithm, which enables it to automatically learn and discover laws and patterns from data, and then use these laws and patterns to perform prediction, classification, clustering, and dimensionality reduction tasks, preferably a deep learning model, combined with modeling and simulation performed by an artificial intelligence algorithm, and regression learning using a neural network tool;

[0134] Micro-motion type identification: Where e is the base of natural logarithms; x 1 …x n is the eigenvector X, β 0...β n is the model parameter β, and P is the type of micro-action to be identified, such as 0 for no action, 1 for turning over, and 2 for grasping.

[0135] In this embodiment, in step S23, during the classification model training, the error between the predicted value and the true value is quantified: Among them, Loss is the cross entropy loss function; y i is the true label of the i-th sample; σ(z i ) is the predicted probability of the i-th sample; N is the number of samples;

[0136] Use an optimization algorithm to minimize the loss function, adjust the parameters of the model, and calculate the gradient of the loss function with respect to each parameter: Among them, w is the weight;

[0137] Where b is the bias;

[0138] Update the parameters using the gradient descent method: Among them, α is the learning rate, which controls the step size of parameter update.

[0139] In some specific embodiments, the step of S3 includes the following sub-steps:

[0140] S31, acquiring infrared monitoring video images of the baby in real time, and converting each frame of the video image into a grayscale image;

[0141] S32, reducing each frame of grayscale image to a fixed size (e.g., 8×8 pixels, for removing high-frequency detail information of the image and retaining low-frequency information), calculating the grayscale average of all pixels in the reduced image, and comparing the grayscale value of each pixel with the average to obtain a hash value for each frame of image;

[0142] S33, calculating the Hamming distance between the hash values ​​of two consecutive frames of images, setting the Hamming distance threshold, and judging the similarity of the two frames of images and whether there is micro-motion according to the size of the Hamming distance and the threshold;

[0143] S34, randomly selecting a frame from the video sequence as the starting frame, and then calculating the Hamming distance between the subsequent frames and the starting frame frame by frame, when the Hamming distance between a certain frame image and the starting frame is less than a threshold and the number of frames between them is greater than half of the video period, the frame is used as the ending frame, and the frame sequence between the starting frame and the ending frame is a key frame sequence;

[0144] S35, extracting the key frame sequence determined to have micro-motions for micro-motion recognition.

[0145] It should be noted that by converting each frame image in the video into a grayscale image, reducing it to a fixed size, calculating the grayscale average, generating a hash value, and quickly judging the similarity between two frames of images through the Hamming distance, the amount of calculation is reduced, the processing speed is improved, the key frame sequence containing complete action information is effectively extracted, the amount of data is reduced, and the efficiency of subsequent processing is improved.

[0146] In this embodiment, in step S32, the grayscale average value of the pixel is calculated: Where, μ is the grayscale average of all pixels in the reduced image; C and D are the number of rows and columns of the reduced image, respectively; I(i, j) is the grayscale value of the reduced image at position (i, j);

[0147] Compare the grayscale value of each pixel of the image with the average value μ, and let the generated binary string be B, where B k is the kth bit of the string (k = 0, 1, ..., 63). For each pixel I (i, j), calculate its corresponding binary value B k : Where k is the index of the pixel in the 64-bit string, which can be determined according to the size of the image and the traversal order. For example, if the image is an 8×8 block, k can be directly determined by the row and column indices i and j of the pixel through some mapping (e.g., k=i×8+j);

[0148] Convert the generated 64-bit binary string B to a decimal hash value:

[0149]

[0150] In this embodiment, in step S33, the Hamming distance between the hash values ​​of two frames of images is: Where HD is the Hamming distance between the hash values ​​of two consecutive frames; L is the length of the hash value, which is 64 bits; U 1 (i) and U 2 (i) are the i-th digits of the two hash values;

[0151] Judging similarity and micro-movements: Among them, S is the judgment result; T is the Hamming distance threshold; 0 means the baby has no micro-movement; 1 means the baby has micro-movement.

[0152] In this embodiment, in step S34, the starting frame F start With the termination frame F end Between, key frame sequence extraction: HD i <T and Where i is the index of the current frame; start is the index of the starting frame; J is the video period; the starting frame F start With the termination frame Fend The frame sequence between them is a key frame sequence.

[0153] In some specific embodiments, the step of S4 includes the following sub-steps:

[0154] S41, extracting optical flow features from the extracted continuous frame images, including the optical flow calculation, optical flow histogram and optical flow amplitude features involved;

[0155] S42, enhancing the edge blurred image of the continuous frame image by using the extracted optical flow features, and fusing the optical flow features with the grayscale features and edge features of the infrared image to obtain a comprehensive feature vector;

[0156] S43, inputting the extracted feature vector into a classification model to identify the type of micro-motion;

[0157] S44, the identified micro-movement type is reminded through the user terminal device, and the identified micro-movement type and time are recorded, and an analysis report of the baby's behavior pattern is provided to the user to help the user better understand the baby's behavior habits and health status.

[0158] In this embodiment, in step S41, the step of extracting the optical flow feature is consistent with the steps in S13, S14 and S15.

[0159] In this embodiment, in step S42, the step of enhancing the edge blurred image and fusing the optical flow feature with the grayscale feature and edge feature of the infrared image is consistent with the step in S16.

[0160] In this embodiment, in step S43, if the classification model outputs "roll over", it is considered that the baby has performed a rolling action; if the classification model outputs "grasp", it is considered that the baby has performed a grasping action, etc.

[0161] In this embodiment, in step S44, the reminder method includes but is not limited to text prompts, sound prompts or vibration prompts; the analysis report includes but is not limited to the frequency, duration and time distribution of each action.

[0162] like Figure 2 As shown, an intelligent monitoring system for an intelligent monitoring method for caring for a baby comprises:

[0163] An image collection module, used to collect continuous frame infrared images under different micro-motions, as well as real-time infrared monitoring videos of infants;

[0164] Image processing module, used for image preprocessing and grayscale processing;

[0165] A feature extraction module is used to extract feature vectors of continuous frame infrared images;

[0166] A classification model is used to identify the type of infant micro-movements based on the extracted feature vectors;

[0167] A similarity analysis module is used to perform similarity analysis of consecutive frames on the acquired real-time infrared monitoring video images of infants;

[0168] A continuous frame image extraction module is used to determine the range of continuous frames of micro-motions based on similarity analysis, and to extract continuous frame images with micro-motions;

[0169] The reminder analysis module is used to remind and analyze the identified micro-action types on the user side.

[0170] It should be noted that the intelligent monitoring system for caring for babies can implement the steps in the intelligent monitoring method for caring for babies in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment, which will not be repeated here.

[0171] The above is based on the ideal embodiment of the present invention. Through the above description, it is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is limited by the attached claims rather than the above description, and it is intended to include all changes within the meaning and scope of the equivalent elements of the claims. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0172] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. An intelligent monitoring method for baby care, characterized in that: The following steps are involved: S1, collecting continuous frame infrared images of infants under different micro-movements, preprocessing the continuous frame infrared images, and extracting feature vectors of the continuous frame infrared images; S2. Based on the feature vector, a classification model for identifying the types of infant micro-movements is constructed. A large number of labeled micro-movement data sets are used to train the classification model. The parameters of the classification model are adjusted using an optimization algorithm to ensure accurate recognition of the classification model. S3, obtaining a real-time infrared monitoring video image of the baby, performing grayscale processing on the video image, performing similarity analysis on continuous frames in the video image, determining the range of continuous frames of micro-motions according to the similarity analysis, and extracting continuous frame images with micro-motions; S4. Extract feature vectors from the extracted continuous frame images and input them into a classification model to identify the types of infant micro-movements. The identified micro-movement types are reminded and analyzed on the user side to realize intelligent monitoring of infant care.

2. The intelligent monitoring method for baby care according to claim 1, characterized in that: The step S1 includes the following sub-steps: S11, collect continuous frame infrared images of infants with different micro-movements; S12, converting the collected continuous frame infrared images into grayscale images, and performing noise reduction processing to ensure image quality; S13, for each continuous frame interval of the micro-motion, calculating the horizontal and vertical optical flows of each frame of the infrared video, and obtaining the motion vector of each pixel between the continuous frames; including the motion direction and speed involved; S14, dividing the direction of the optical flow vector into several direction intervals, counting the number of optical flow vectors in each direction interval to form an optical flow histogram, and extracting the amplitude characteristics of the optical flow, counting the distribution of the optical flow amplitude, and obtaining information about the motion intensity; S15, perform multi-scale transformation on the infrared image, calculate the optical flow features at each scale, capture the motion information at different scales, and fuse the optical flow features at different scales to form a comprehensive optical flow feature representation; S16. Perform edge detection on the infrared image using an edge detection algorithm to obtain an edge map, combine the edge information of the edge map with the motion information of the optical flow feature map, enhance the optical flow features of the edge area, and fuse the optical flow features with the grayscale features and edge features of the infrared image to obtain a comprehensive feature vector.

3. The intelligent monitoring method for baby care according to claim 2, characterized in that: In the step S13, calculating the horizontal and vertical optical flows of each frame of infrared video includes the following steps: S131, assuming that the image brightness is constant, let I(x, y, t) be the brightness value of the image at position (x, y) and time t, I(x, y, t) = I(x + Δx, y + Δy, t + Δt); S132. Perform Taylor expansion on the brightness value of the image at the position and time, ignoring the higher-order terms: I(x,y,t)≈I(x+Δx,y+Δy,t+Δt)+I x Δx+I y Δy+I t Δt; where I x ,I y and I t are the partial derivatives of the image in the x, y and t directions respectively; due to the constant brightness, it is simplified to: I x Δx+I y Δy+I t Δt=0; S133, assuming Δx=u and Δy=v, where u is the horizontal speed of each pixel and v is the vertical speed of each pixel, then I x u+I y v+I t Δt=0; Matrix form: S134. Within the local window, the motion is uniform, and the matrix form is: Where n is the number of pixels in the local window; by solving the least squares method, we get: Where A is the coefficient matrix, b is a constant term vector, A T is the transpose of matrix A; S135 , repeat the above steps for each frame image in the continuous frame interval of each micro-motion, and calculate the horizontal and vertical optical flows of each frame image.

4. The intelligent monitoring method for baby care according to claim 2, characterized in that: In the step S14, the following steps are specifically included: S141, the direction of the optical flow vector is divided into 8 intervals, each interval corresponds to an angle range of 45 degrees; S142, for each pixel's optical flow vector Calculate direction θ and amplitude Where, direction θ = arctan2(v,u), arctan2 is the four-quadrant inverse tangent function, and the return value range is [-180,180] degrees; S143, mapping the direction θ to a range of 0 to 360 degrees, determining which direction interval it belongs to, and counting the number of optical flow vectors in each direction interval to form an optical flow histogram; S144. Count the optical flow amplitude values ​​of all pixels to obtain information about motion intensity.

5. The intelligent monitoring method for baby care according to claim 2, characterized in that: In the step S15, the following steps are specifically included: S151, using a Gaussian filter to perform blur processing on the infrared image, removing high-frequency details in the image, and performing a downsampling operation on the blurred image; S152, taking the reduced image as input, repeatedly performing Gaussian blur and downsampling steps to construct the next layer of the Gaussian pyramid until the required Gaussian pyramid level is reached; S153, taking each level image of the Gaussian pyramid as input, for each scale image I s , calculate the optical flow features between adjacent frames and obtain the optical flow histogram H s , capturing motion information at different scales; S154, the optical flow histogram H at each scale s Assign a weight w s , the optical flow histogram H at different scales s Perform weighted summation to obtain the comprehensive optical flow histogram H, H = ∑w s H s .

6. The intelligent monitoring method for baby care according to claim 2, characterized in that: In the step S16, the following steps are specifically included: S161. Calculate the gradient of the image in the x and y directions: Calculate the gradient magnitude M and direction θ: θ=arctan2(G y ,G x ); S162, for each pixel, check the neighboring pixels in the gradient direction, and if the gradient magnitude of the current pixel is not a local maximum, set its gradient magnitude to 0; S163, using two thresholds T high and T low Perform threshold processing on the gradient amplitude; where T high For detecting strong edges, T low Used to detect weak edges; when the gradient amplitude is greater than T high , then the pixel is considered to be a strong edge; when the degree amplitude is T low and T high When the gradient amplitude is less than T low , then the pixel is suppressed; S164, checking the neighboring pixels of the weak edge pixel, if there is a strong edge pixel among the neighboring pixels, retaining the weak edge pixel to obtain an edge map; S165. For the optical flow features u and v of the edge area, weighted enhancement is adopted, that is, for each pixel, when the pixel is an edge pixel in the edge map E, the optical flow features u(x, y) and v(x, y) of the pixel are multiplied by a weight w; S166, the grayscale feature map I, the edge feature map E, the optical flow feature maps u and v are fused to form a comprehensive feature vector, 7. The intelligent monitoring method for baby care according to claim 1, characterized in that: The step S2 includes the following sub-steps: S21, using the extracted feature vector as input feature and the micro-movement type as output target, constructing a classification model capable of identifying the type of infant micro-movement; The identification of the infant micro-movement type adopts a machine learning algorithm, and the micro-movement type identification: Where e is the base of natural logarithms; x1...x n is the eigenvector X, β0∈β n is the model parameter β, P is the type of micro-motion identified; S22, labeling a large amount of infant micro-movement video data, clarifying the specific movement type of the infant in each video, and pre-processing the labeled data; including grayscale conversion and noise reduction processing involved; S23, dividing the processed data into a training set and a validation set, using the training set data to train the classification model, using the validation set data to validate the classification model, using an optimization algorithm to adjust the parameters of the classification model, and through multiple iterations of training, enabling the classification model to accurately identify different types of micro-motions; In the classification model training, the error between the predicted value and the true value is quantified: Among them, Loss is the cross entropy loss function; y i is the true label of the i-th sample; σ(z i ) is the predicted probability of the i-th sample; N is the number of samples; Use an optimization algorithm to minimize the loss function, adjust the parameters of the model, and calculate the gradient of the loss function with respect to each parameter: Among them, w is the weight; Where b is the bias; Update the parameters using the gradient descent method: Among them, α is the learning rate, which controls the step size of parameter update.

8. The intelligent monitoring method for baby care according to claim 1, characterized in that: The step S3 includes the following sub-steps: S31, acquiring infrared monitoring video images of the baby in real time, and converting each frame of the video image into a grayscale image; S32, reducing each frame of grayscale image to a fixed size, calculating the grayscale average value of all pixels in the reduced image, and comparing the grayscale value of each pixel with the average value to obtain a hash value of each frame of image; Calculate the grayscale average of the pixel: Where, μ is the grayscale average of all pixels in the reduced image; C and D are the number of rows and columns of the reduced image, respectively; I(i, j) is the grayscale value of the reduced image at position (i, j); Compare the grayscale value of each pixel of the image with the average value μ, and let the generated binary string be B, where B k is the kth bit of the string (k = 0, 1, ..., 63). For each pixel I (i, j), calculate its corresponding binary value B k : Where k is the index of the pixel in the 64-bit string; Convert the generated 64-bit binary string B to a decimal hash value: S33, calculating the Hamming distance between the hash values ​​of two consecutive frames of images, setting the Hamming distance threshold, and judging the similarity of the two frames of images and whether there is micro-motion according to the size of the Hamming distance and the threshold; The Hamming distance between the hash values ​​of the two frames of images: Where HD is the Hamming distance between the hash values ​​of two consecutive frames of images; L is the length of the hash value, i.e., 64 bits; U1(i) and U2(i) are the i-th bits of the two hash values ​​respectively; Judging similarity and micro-movements: Among them, S is the judgment result; T is the Hamming distance threshold; 0 means the baby has no micro-movement; 1 means the baby has micro-movement; S34, randomly selecting a frame from the video sequence as the starting frame, and then calculating the Hamming distance between the subsequent frames and the starting frame frame by frame, when the Hamming distance between a certain frame image and the starting frame is less than a threshold and the number of frames between them is greater than half of the video period, the frame is used as the ending frame, and the frame sequence between the starting frame and the ending frame is a key frame sequence; The starting frame F start With the termination frame F end Between, key frame sequence extraction: HD i <T and Where i is the index of the current frame; start is the index of the starting frame; J is the video period; the starting frame F start With the termination frame F end The frame sequence between is a key frame sequence; S35, extracting the key frame sequence determined to have micro-motions for micro-motion recognition.

9. The intelligent monitoring method for baby care according to claim 1, characterized in that: The step S4 includes the following sub-steps: S41, extracting optical flow features from the extracted continuous frame images, including the optical flow calculation, optical flow histogram and optical flow amplitude features involved; S42, enhancing the edge blurred image of the continuous frame image by using the extracted optical flow features, and fusing the optical flow features with the grayscale features and edge features of the infrared image to obtain a comprehensive feature vector; S43, inputting the extracted feature vector into a classification model to identify the type of micro-motion; S44, the identified micro-movement type is reminded through the user terminal device, and the identified micro-movement type and time are recorded, and an analysis report of the baby's behavior pattern is provided to the user to help the user better understand the baby's behavior habits and health status.

10. An intelligent monitoring system based on an intelligent monitoring method for caring for infants according to any one of claims 1 to 9, characterized in that: include: An image collection module, used to collect continuous frame infrared images under different micro-motions, as well as real-time infrared monitoring videos of infants; Image processing module, used for image preprocessing and grayscale processing; A feature extraction module is used to extract feature vectors of continuous frame infrared images; A classification model is used to identify the type of infant micro-movements based on the extracted feature vectors; A similarity analysis module is used to perform similarity analysis of consecutive frames on the acquired real-time infrared monitoring video images of infants; A continuous frame image extraction module is used to determine the range of continuous frames of micro-motions based on similarity analysis, and to extract continuous frame images with micro-motions; The reminder analysis module is used to remind and analyze the identified micro-action types on the user side.

Images