Infant nursing system and method for obstetrics and gynecology department

Abstract

The invention discloses an infant nursing system and method for the obstetrics and gynecology department, and mainly relates to the technical field of safety monitoring. Comprising a face image frame sequence obtaining module used for arranging an infrared camera in a newborn nursing room and performing face image acquisition by using the infrared camera to obtain a face image frame sequence; the micro-expression key frame sequence obtaining module is used for traversing the facial image frame sequence to perform micro-expression key frame extraction so as to obtain a micro-expression key frame sequence; the micro-expression feature vector sequence obtaining module is used for extracting spatio-temporal features to obtain a micro-expression feature vector sequence; and the target nursing response scheme obtaining module is used for carrying out nursing response scheme matching according to the micro-expression feature vector sequence to obtain a target nursing response scheme. The method has the advantages that the technical problems that in the prior art, multi-mode nursing information cannot be efficiently and accurately integrated, and the fitting degree of nursing and the actual situation is low are solved, and the technical effect of improving the nursing reliability is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of safety guardianship, and particularly to an obstetrics and gynecology infant care system and method. Background Art

[0002] The existing obstetrics and gynecology infant care mainly relies on manual monitoring by nurses or family members. Usually, the state of the infant is judged by observing external behaviors such as the infant's expression, cry, and body movement, and care is provided based on experience. However, this method has a large degree of subjectivity and is easily affected by the experience, attention, and environmental factors of the guardians, resulting in unstable judgment results.

[0003] Currently, there have been studies attempting to use computer vision technology to recognize infant facial expressions, but these methods usually rely on adult facial expression recognition models and cannot effectively adapt to the particularity of infant facial expressions, such as small amplitude of facial expression changes and few expression categories. In addition, existing emotion recognition systems often only rely on data in a single modality, such as facial images or cries, and fail to comprehensively analyze the multi-modal information of infants, resulting in limited recognition accuracy and inability to provide accurate care response plans. Summary of the Invention

[0004] This application provides an obstetrics and gynecology infant care system and method for solving the technical problems in the prior art that multi-modal care information cannot be efficiently and accurately integrated and the fit between care and the actual situation is low.

[0005] In view of the above problems, this application provides an obstetrics and gynecology infant care system and method.

[0006] In the first aspect of this application, an obstetrics and gynecology infant care system is provided. The system includes:

[0007] An infrared camera is arranged in the neonatal care room, and the infrared camera is used to collect facial images to obtain a sequence of facial image frames;

[0008] Traverse the sequence of facial image frames to extract micro-expression key frames, and obtain a sequence of micro-expression key frames;

[0009] Extract spatio-temporal features from the sequence of micro-expression key frames to obtain a sequence of micro-expression feature vectors;

[0010] Match a care response plan according to the sequence of micro-expression feature vectors to obtain a target care response plan.

[0011] In the second aspect of this application, an obstetrics and gynecology infant care method is provided. The method includes:

[0012] A facial image frame sequence acquisition module is configured to deploy an infrared camera in a neonatal care room, use the infrared camera to collect facial images, and obtain a facial image frame sequence;

[0013] A micro-expression key frame sequence acquisition module is configured to traverse the facial image frame sequence to extract micro-expression key frames and obtain a micro-expression key frame sequence;

[0014] A micro-expression feature vector sequence acquisition module is configured to perform spatio-temporal feature extraction on the micro-expression key frame sequence to obtain a micro-expression feature vector sequence;

[0015] A target care response plan acquisition module is configured to match a care response plan according to the micro-expression feature vector sequence and obtain a target care response plan.

[0016] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0017] In this application, an infrared camera is deployed in a neonatal care room, and the infrared camera is used to collect facial images to obtain a facial image frame sequence. Then, the facial image frame sequence is traversed to extract micro-expression key frames to obtain a micro-expression key frame sequence. Furthermore, spatio-temporal feature extraction is performed on the micro-expression key frame sequence to obtain a micro-expression feature vector sequence. Then, a care response plan is matched according to the micro-expression feature vector sequence to obtain a target care response plan. The technical effect of improving the quality of infant care and obtaining a care plan that meets the actual care needs is achieved. Description of the Drawings

[0018] Att Figure 1 is a schematic structural diagram of an obstetrics and gynecology infant care system provided by an embodiment of the present invention.

[0019] Att Figure 2 is a schematic flowchart of an obstetrics and gynecology infant care method provided by an embodiment of the present invention.

[0020] Reference numerals shown in the drawings:

[0021] Facial image frame sequence acquisition module 11, micro-expression key frame sequence acquisition module 12, micro-expression feature vector sequence acquisition module 13, target care response plan acquisition module 14. Detailed Embodiments

[0022] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application. It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices.

[0023] Embodiment 1, as shown in the appendix Figure 1 The present application provides an obstetrics and gynecology infant care system, and the system includes:

[0024] A facial image frame sequence acquisition module 11, configured to deploy an infrared camera in the neonatal care room, use the infrared camera to collect facial images, and obtain a facial image frame sequence;

[0025] In a possible embodiment, an infrared camera is deployed in the neonatal care room. Preferably, an infrared camera with high resolution and low noise is selected to ensure that the facial expressions of the baby can be clearly captured in a low-light environment. During deployment, the camera should avoid direct sunlight from strong light sources as much as possible and be installed above or on the side of the baby's crib to obtain stable and complete facial images. The infrared camera adopts a real-time video stream acquisition method and continuously acquires a facial image frame sequence at a fixed frame rate (such as 30fps). Considering the head movement of the baby, to ensure data quality,

[0026] Exemplarily, assuming that the system acquires video at a frame rate of 30fps, 30 frames of facial images are extracted per second to obtain the facial image frame sequence. Preferably, the system can set a time window (such as 5 seconds) to cache 150 frames of images for subsequent micro-expression analysis. The facial image frame sequence serves as the basis for subsequent micro-expression key frame extraction and emotion recognition, ensuring data quality while also providing stable input for subsequent analysis.

[0027] A micro-expression key frame sequence acquisition module 12, configured to traverse the facial image frame sequence to extract micro-expression key frames and obtain a micro-expression key frame sequence;

[0028] Further, the micro-expression key frame sequence acquisition module 12 is configured to perform the following steps:

[0029] Traverse the facial image frame sequence for grayscale processing to obtain a facial image grayscale frame sequence;

[0030] Calculate the magnitude of the optical flow field vector for adjacent frames of the grayscale frame sequence of the facial image based on the Farneback optical flow method to obtain the change curve of the magnitude of the optical flow field vector;

[0031] Perform nearest neighbor clustering analysis on the change curve of the magnitude of the optical flow field vector to obtain the clusters of the magnitude of the optical flow field vector;

[0032] Extract the micro-expression key frames from the grayscale frame sequence of the facial image based on the clusters of the magnitude of the optical flow field vector to obtain the micro-expression key frame sequence.

[0033] In a possible embodiment, since the facial image frame sequence is a sequence of consecutive images and there are many redundant images, in order to accurately analyze the micro-expression changes of the baby's face reflected by the images, it is necessary to extract the micro-expression key frames from the facial image frame sequence to obtain the micro-expression key frame sequence.

[0034] First, traverse the facial image frame sequence and perform grayscale processing on it to remove color information and reduce computational complexity, obtaining the grayscale frame sequence of the facial image. Then, calculate the optical flow field between adjacent frames based on the Farneback optical flow method, extract the motion vectors of each pixel point, and calculate their magnitudes, thereby obtaining the change curve of the magnitude of the optical flow field vector. Next, process the change curve of the magnitude of the optical flow field vector through nearest neighbor clustering analysis to find the regions with large changes and form clusters of the magnitude of the optical flow field vector, which can reflect the small change regions of facial expressions. Finally, based on these magnitude clusters, extract key frames from the grayscale frame sequence of the facial image to form the micro-expression key frame sequence for subsequent analysis of the baby's emotional characteristics and matching of care plans. The role of this process is to accurately extract the critical moments of the baby's micro-expressions, reduce redundant frames, and improve the accuracy and computational efficiency of subsequent emotion recognition and response plan matching.

[0035] In a possible embodiment, the change curve of the magnitude of the optical flow field vector is the change trend curve of the magnitude of the optical flow field vector in the time dimension, which is used to detect the grayscale frames of the facial image where micro-expressions occur. The grayscale frame sequence of the facial image refers to a sequence of consecutive image frames after grayscale processing, where each frame is a grayscale image, only retaining the brightness information of the facial features and removing color interference.

[0036] Preferably, each frame of the collected facial image frames usually consists of three channels: red (R), green (G), and blue (B). The core of grayscale conversion is to combine these three channels in a certain way while ensuring the clarity of the image. For example, the weighted average method (Gray = 0.299R + 0.587G + 0.114B) is used to convert the color information to obtain a single-channel grayscale image. By performing grayscale conversion, the grayscale frame sequence of the facial image is obtained. This step can reduce the data complexity, reduce the computational amount, and at the same time enhance the recognizability of key information such as facial contours and detailed textures, providing more stable input data for subsequent optical flow calculation. In addition, grayscale conversion can eliminate the interference caused by color changes, making subsequent micro-expression recognition more accurate.

[0037] Further, the micro-expression key frame sequence obtaining module 12 is used to perform the following steps:

[0038] Based on the Farneback optical flow method, in the order from front to back in time, perform dense optical flow calculation on adjacent frames in the grayscale frame sequence of the facial image to obtain an adjacent frame optical flow field sequence, where each adjacent frame optical flow field in the adjacent frame optical flow field sequence is the optical flow field between the current frame and the previous frame;

[0039] Traverse the adjacent frame optical flow field sequence to perform horizontal component and vertical component calculations to obtain an adjacent frame optical flow field horizontal component set sequence and an adjacent frame optical flow field vertical component set sequence;

[0040] Use the optical flow field vector amplitude calculation function to calculate the adjacent frame optical flow field horizontal component set sequence and the adjacent frame optical flow field vertical component set sequence to obtain an optical flow field vector amplitude sequence;

[0041] Fit the optical flow field vector amplitude sequence to construct the optical flow field vector amplitude change curve.

[0042] Further, the optical flow field vector amplitude calculation function is:

[0043]

[0044] where M ROI is the optical flow field vector amplitude, N is the number of pixel points in the grayscale frame of the facial image, N is a positive integer, dx(x i ,y i ) is the horizontal component of the adjacent frame optical flow field of the i-th pixel point in the grayscale frame of the facial image, and dy(x i ,y i ) is the vertical component of the adjacent frame optical flow field of the i-th pixel point in the grayscale frame of the facial image.

[0045] In a possible embodiment, the Farneback optical flow method is a dense optical flow estimation algorithm that can calculate the pixel-level motion information between adjacent frames in a video sequence. The Farneback optical flow method can provide an optical flow vector for each pixel and is applicable to global motion analysis. The calculation of the magnitude of the optical flow field vector between adjacent frames is to calculate the motion vector of each pixel point between two adjacent frames, and this vector is composed of a horizontal component (dx) and a vertical component (dy). The sequence of the optical flow field vectors between adjacent frames refers to the motion vector field of all pixel points in the current frame and the previous frame, where the motion of each pixel point is represented by a vector (dx, dy), and the magnitude and direction of this vector represent the motion situation of this pixel point. Among them, each optical flow field between adjacent frames in the sequence of the optical flow field vectors between adjacent frames is the optical flow field between the current frame and the previous frame. The change curve of the magnitude of the optical flow field vector represents the trend of the motion intensity of the facial area changing over time in the entire image frame sequence, and can be used to detect subtle facial motions, such as the occurrence of micro-expressions.

[0046] The horizontal component (dx) of the optical flow field between adjacent frames and the vertical component (dy) of the optical flow field between adjacent frames respectively represent the motion of the pixel point in the horizontal direction and the vertical direction. dx is the motion component in the x direction, and dy is the motion component in the y direction. The magnitude of the optical flow field vector is used to measure the overall motion intensity of the grayscale frame of the facial image, and the calculation method is to average the magnitudes of the optical flow vectors of all pixel points.

[0047] Preferably, any two adjacent frames in the sequence of grayscale frames of the facial image are extracted, and each pair of frames is used as a group of inputs, and the Farneback optical flow calculation is performed sequentially in chronological order. Specifically, for the current frame and the previous frame, the motion vectors (dx, dy) of all pixel points in the image are calculated to generate the optical flow field of the current frame relative to the previous frame. This process continues until all adjacent frames are calculated, so as to obtain a complete sequence of the optical flow fields between adjacent frames. By performing running vector analysis, the motion information of the facial area, especially the subtle muscle movements, can be extracted. For example, when a baby smiles, the muscles at the corners of the mouth and the cheeks will rise slightly, and this change can be captured by the direction and magnitude of the vectors in the optical flow field. By analyzing the entire sequence of the optical flow fields, the motion patterns of the facial muscles can be identified, providing basic data for the subsequent extraction of key frames of micro-expressions.

[0048] The system sequentially traverses each frame in the sequence of the optical flow fields, and decomposes the motion vectors of all pixel points therein, respectively extracting the dx component in the x direction and the dy component in the y direction. The dx and dy components of each frame form a set, and over time, these sets are arranged in sequence to form a complete sequence of the sets of the horizontal components of the optical flow fields between adjacent frames and a complete sequence of the sets of the vertical components of the optical flow fields between adjacent frames.

[0049] By decomposing complex optical flow information into independent horizontal and vertical motion features, subsequent motion pattern analysis becomes easier. For example, when an infant frowns due to discomfort, it mainly occurs in the vertical direction (negative dy value) in the forehead area; while when the infant shows a happy expression, the corners of the mouth may stretch to both sides, mainly occurring in the horizontal direction (positive dx value). This decomposition helps to more accurately capture and analyze the subtle changes in facial micro-expressions, providing data support for the extraction of key frames of micro-expressions in the future.

[0050] Preferably, the optical flow field vector amplitude calculation function is used to quantify the motion intensity of the facial area. By calculating the motion vector amplitudes of all pixel points in each frame, a sequence of optical flow field vector amplitudes changing with time is obtained. The optical flow field vector amplitude change curve refers to the smooth curve obtained by fitting the sequence of optical flow field vector amplitudes, which is used to describe the change trend of the optical flow field vector amplitude with time and reflect the dynamic characteristics of facial expression motion. Optionally, the moving average method is used to fit the sequence of optical flow field vector amplitudes. By calculating the local average value according to the sliding window preset by those skilled in the art, short-term fluctuations are smoothed to make the curve more stable.

[0051] Exemplarily, when the sequence of optical flow field vector amplitudes includes 10 optical flow field vector amplitudes, which are: 0.2, 0.3, 0.5, 0.8, 1.2, 1.5, 1.0, 0.6, 0.3, 0.1. The sliding window preset by those skilled in the art is 3. At this time, the sliding window calculation is performed, and the obtained smooth data point sequence is: 0.33, 0.53, 0.83, 1.17, 1.23, 1.03, 0.63, 0.33. Furthermore, with time as the horizontal coordinate axis and the optical flow field vector amplitude as the vertical coordinate axis, a coordinate system is constructed. The smooth data point sequence is input into the coordinate system in sequence to obtain a sequence of coordinate points, and the sequence of coordinate points is connected in sequence to obtain the optical flow field vector amplitude change curve after fitting.

[0052] Furthermore, the micro-expression key frame sequence obtaining module 12 is used to perform the following steps:

[0053] Extract the local maxima in the optical flow field vector amplitude change curve to obtain a set of nearest neighbor cluster center points;

[0054] Respectively starting from the set of nearest neighbor cluster center points, perform nearest neighbor cluster analysis according to the preset amplitude difference threshold to obtain the optical flow field vector amplitude clusters.

[0055] In one embodiment of the present application, a local maximum refers to a point whose value is greater than the values of other points within a certain neighborhood, representing a significant peak point in the amplitude change curve of the optical flow field vector. The central point of the nearest neighbor cluster is the basis for clustering. By combining a preset amplitude difference threshold, the neighborhood belonging to each central point of the nearest neighbor cluster can be delimited, thereby obtaining the optical flow field vector set of each central point of the nearest neighbor cluster, and summarizing them to obtain the optical flow field vector amplitude cluster. Trend-based Filtering refers to filtering out the amplitude clusters that conform to the trend change by calculating the mean value of the amplitude clusters, and excluding outliers or noise points. The preset amplitude threshold is the minimum amplitude difference between the optical flow field vector amplitude and the central point of the nearest neighbor cluster when it can be delimited into the neighborhood of the central point of the nearest neighbor cluster by those skilled in the art, which is used to screen effective micro-expression key frames to ensure that the selected key frames have sufficient motion amplitude to represent micro-expression changes.

[0056] Preferably, traverse the amplitude change curve of the optical flow field vector to extract local maxima, and use the coordinate points where the local maxima are located as the central points of the nearest neighbor clusters, thereby obtaining the set of central points of the nearest neighbor clusters. Among them, the set of central points of the nearest neighbor clusters are respectively the coordinate points where the amplitude of the optical flow field vector is greater than the amplitudes of the adjacent left and right coordinate points. Furthermore, starting from each central point of the nearest neighbor cluster, calculate the amplitude difference between it and the neighboring points. If the difference is less than the preset amplitude difference threshold, then include this point in the same cluster until a point with a large amplitude drop or exceeding the threshold range is encountered, thereby obtaining the optical flow field vector amplitude cluster.

[0057] Further, the micro-expression key frame sequence obtaining module 12 is used to perform the following steps:

[0058] Traverse and calculate the mean value of the optical flow field vector amplitude cluster to obtain the set of optical flow field vector amplitude means;

[0059] Respectively, taking the set of optical flow field vector amplitude means as the trend starting point, perform trend-based filtering on the optical flow field vector amplitude cluster to obtain the filtered optical flow field vector amplitude cluster;

[0060] Calculate the mean value of the filtered optical flow field vector amplitude cluster to obtain the set of filtered optical flow field vector amplitude means;

[0061] Judge whether the set of filtered optical flow field vector amplitude means is greater than or equal to the preset amplitude threshold. If so, sort the multiple facial image grayscale frames corresponding to the optical flow field vector amplitude cluster in chronological order to obtain the micro-expression key frame sequence.

[0062] In a possible embodiment, traverse and calculate the mean value of the optical flow field vector amplitude clusters to obtain the optical flow field vector amplitude mean value set. Among them, the optical flow field vector amplitude mean value set reflects the average level of the optical flow field vector amplitude sets of each neighboring cluster center in the optical flow field vector amplitude clusters. Furthermore, taking the optical flow field vector amplitude mean value set as the trend starting point, construct a neighborhood of the trend starting point according to a preset screening step size to obtain a set of trend starting point neighborhoods. Among them, each trend starting point neighborhood corresponds to an optical flow field vector amplitude mean value. The preset screening step size is the amplitude of a single expansion during the centralized trend screening preset by those skilled in the art.

[0063] Preferably, respectively count the number of optical flow field vector amplitudes in the set of trend starting point neighborhoods, and divide the statistical result by twice the preset screening step size to obtain the density of the set of trend starting point neighborhoods, that is, the set of trend neighborhood densities. Furthermore, expand the left and right ends of the set of trend starting point neighborhoods outward according to the preset screening step size to obtain a set of first-order diffusion neighborhoods of the trend starting point. Based on the same principle as obtaining the set of trend neighborhood densities, calculate the density of the set of first-order diffusion neighborhoods of the trend starting point to obtain a set of first-order diffusion neighborhood densities of the trend starting point.

[0064] Judge whether the set of first-order diffusion neighborhood densities of the trend starting point is greater than or equal to the corresponding set of trend neighborhood densities. If so, then based on the preset screening step size, expand the set of first-order diffusion neighborhoods of the trend starting point outward again to obtain a set of second-order diffusion neighborhoods of the trend starting point, and continue the expansion until the difference in neighborhood densities between two adjacent expansions is less than or equal to a preset difference, then stop the expansion to obtain a set of target diffusion neighborhoods of the trend starting point. And summarize the set of target diffusion neighborhoods of the trend starting point to obtain the screened optical flow field vector amplitude clusters.

[0065] Furthermore, respectively calculate the mean value of the screened optical flow field vector amplitude clusters to obtain a set of screened optical flow field vector amplitude mean values that can reflect the amplitude situation of each screened optical flow field vector amplitude set. The preset amplitude threshold is the minimum amplitude corresponding to the micro-expression key frame preset by those skilled in the art. Judge whether the set of screened optical flow field vector amplitude mean values is greater than or equal to the preset amplitude threshold. If so, it indicates that the corresponding facial image grayscale frame is a micro-expression key frame. At this time, sort the multiple facial image grayscale frames corresponding to the optical flow field vector amplitude clusters in chronological order to obtain the micro-expression key frame sequence.

[0066] A micro-expression feature vector sequence obtaining module 13, configured to perform spatio-temporal feature extraction on the micro-expression key frame sequence to obtain a micro-expression feature vector sequence;

[0067] In a possible embodiment, a spatio-temporal feature extractor is constructed, and the spatio-temporal feature extractor is used to extract features from the micro-expression key frame sequence to obtain a micro-expression feature vector sequence. Among them, the spatio-temporal feature extractor refers to a model or algorithm that can capture both temporal and spatial information at the same time. In micro-expression analysis, spatial features refer to local textures and shape changes of the face in a single-frame image (such as the slight upward curve of the corners of the mouth and the changes in fine wrinkles around the eyes), while temporal features refer to the dynamic changes of the face between multiple frames (such as the start, development, and end processes of micro-expressions). The micro-expression feature vector sequence is a numerical high-dimensional vector representation obtained by converting the key frame sequence after feature extraction, and can be used for subsequent classification, analysis, and other tasks.

[0068] Preferably, a plurality of sample micro-expression key frames and a plurality of sample micro-expression feature vectors are obtained as training data, and the training data is used to perform supervised training on a framework constructed based on a feed-forward neural network, and the network parameters of the framework are updated according to the output results during training until the training converges, and the trained spatio-temporal feature extractor is obtained.

[0069] The target care response plan obtaining module 14 is configured to match a care response plan according to the micro-expression feature vector sequence to obtain a target care response plan.

[0070] Further, the target care response plan obtaining module 14 is configured to perform the following steps:

[0071] Collect sound signals using a microphone disposed at the care bed of the target infant to obtain a sound signal sequence;

[0072] Extract abnormal features from the sound signal sequence to obtain a sound anomaly feature set;

[0073] Based on the sound anomaly feature set, correct the target care response plan to obtain a corrected care response plan.

[0074] In one embodiment, the target care response plan refers to an intervention measure taken for the object under care (such as an infant) in a specific state, such as adjusting the environmental light, playing soothing music, or notifying the caregiver. The micro-expression feature vector sequence is generated by the previous steps and represents the dynamic micro-expression features of the infant's face, and these features can be used to infer the infant's emotions or needs (such as restlessness, hunger, sleepiness).

[0075] Preferably, a pre - constructed support vector machine (SVM) is used to classify micro - expression features, such as distinguishing states like "slightly uneasy", "anxious crying", "comfortable and relaxed", etc. Then, the most suitable care plan is retrieved from a predefined care plan database. For example, the care plan corresponding to slightly uneasy is slight cradle vibration + soft background music, the care plan corresponding to anxious crying is increasing the amplitude of cradle vibration + simulating the mother's heartbeat sound, and the care plan corresponding to drowsiness is turning off strong light sources + providing a warm sense of wrapping. Those skilled in the art set care plans corresponding to different micro - expression feature vectors according to the actual situation and store them in the database to obtain the care plan database. Furthermore, the matched target care response plan is used as the output to guide intelligent care devices (such as intelligent baby cribs, automatic soothing systems) to take corresponding measures. The role of this process is to achieve automated and intelligent monitoring and soothing of the baby's state based on facial micro - expressions, improving the accuracy and response efficiency of care.

[0076] In one embodiment, sound signal acquisition refers to recording the sounds around the target baby through a microphone, such as crying sounds, laughing sounds, babbling sounds, or breathing sounds. The sound signal sequence is stored in the form of time - series data after these sounds are collected for subsequent analysis. Abnormal feature extraction mainly refers to performing feature analysis on the sound signal to identify abnormal situations, such as the frequency, intensity, and duration of crying sounds. The set of sound abnormal features is the abnormal patterns extracted from the collected sound signals, such as suddenly increased crying, intermittent sobbing, long - term silence, etc., which may indicate that the baby has a need or discomfort. The modified care response plan refers to adjusting the plan based on the sound abnormal features on the basis of the existing care plan to more accurately meet the baby's needs, such as enhancing the soothing intensity or notifying the caregiver.

[0077] Preferably, a convolutional neural network is used to construct a sound abnormal feature extractor. By obtaining multiple sample sound signal sequences and their corresponding multiple sample sound abnormal feature sets as a sample training data set, the sample training data set is evenly divided into n groups of sample training data. Then, the framework constructed based on the convolutional neural network is supervised and trained in turn using the n groups of sample training data, and the network parameters of the framework are corrected according to the results output by each group of sample training data until the training converges, obtaining the trained sound abnormal feature extractor. The trained sound abnormal feature extractor is used to extract abnormal features from the sound signal sequence to obtain the set of sound abnormal features.

[0078] Preferably, search for existing infant care databases and analyze the best care measures corresponding to the current abnormal sound feature set. For example, high-frequency rapid crying + long-term persistence may indicate discomfort (such as pain, eczema). Based on the matching results, modify the target care response plan, such as increasing the intensity of soothing actions (such as increasing the swing amplitude of the cradle), changing the soothing mode (such as switching from light music to simulated mother's heartbeat sound), etc., to obtain the modified care response plan. This achieves the technical effect of ensuring the dynamic adaptation of the care plan, enabling it to be adjusted according to the infant's immediate state, improving the accuracy of intelligent care, and reducing misjudgment.

[0079] In summary, the embodiments of the present application at least have the following technical effects:

[0080] In the present application, an infrared camera is deployed in the neonatal care room, and the infrared camera is used to collect facial images to obtain a facial image frame sequence. Then, the facial image frame sequence is traversed to extract micro-expression key frames to obtain a micro-expression key frame sequence. Furthermore, spatio-temporal features are extracted from the micro-expression key frame sequence to obtain a micro-expression feature vector sequence. Then, a care response plan is matched according to the micro-expression feature vector sequence to obtain a target care response plan. This achieves the technical effect of improving the quality of infant care and obtaining a care plan that meets the actual care needs.

[0081] Embodiment 2, based on the same inventive concept as the obstetrics and gynecology infant care system in the foregoing embodiment, as shown in the appendix Figure 2 The present application provides an obstetrics and gynecology infant care method. The method in the embodiments of the present application is based on the same inventive concept as the system embodiment. Among them, the method includes:

[0082] Deploy an infrared camera in the neonatal care room, and use the infrared camera to collect facial images to obtain a facial image frame sequence;

[0083] Traverse the facial image frame sequence to extract micro-expression key frames to obtain a micro-expression key frame sequence;

[0084] Extract spatio-temporal features from the micro-expression key frame sequence to obtain a micro-expression feature vector sequence;

[0085] Match a care response plan according to the micro-expression feature vector sequence to obtain a target care response plan.

[0086] Furthermore, the method further includes:

[0087] Traverse the facial image frame sequence for grayscale processing to obtain a facial image grayscale frame sequence;

[0088] Calculate the magnitude of the optical flow field vector for adjacent frames of the grayscale frame sequence of the facial image based on the Farneback optical flow method to obtain the curve of the magnitude change of the optical flow field vector;

[0089] Perform a nearest neighbor clustering analysis on the curve of the magnitude change of the optical flow field vector to obtain the clusters of the magnitude of the optical flow field vector;

[0090] Extract the micro-expression key frames from the grayscale frame sequence of the facial image based on the clusters of the magnitude of the optical flow field vector to obtain the micro-expression key frame sequence.

[0091] Furthermore, the method further includes:

[0092] Perform dense optical flow calculation on adjacent frames in the grayscale frame sequence of the facial image in chronological order based on the Farneback optical flow method to obtain the adjacent frame optical flow field sequence, where each adjacent frame optical flow field in the adjacent frame optical flow field sequence is the optical flow field between the current frame and the previous frame;

[0093] Traverse the adjacent frame optical flow field sequence to calculate the horizontal component and the vertical component to obtain the adjacent frame optical flow field horizontal component set sequence and the adjacent frame optical flow field vertical component set sequence;

[0094] Use the optical flow field vector magnitude calculation function to calculate the adjacent frame optical flow field horizontal component set sequence and the adjacent frame optical flow field vertical component set sequence to obtain the optical flow field vector magnitude sequence;

[0095] Fit the optical flow field vector magnitude sequence to construct the curve of the magnitude change of the optical flow field vector.

[0096] Furthermore, the optical flow field vector magnitude calculation function is:

[0097]

[0098] where M ROI is the magnitude of the optical flow field vector, N is the number of pixel points in the grayscale frame of the facial image, N is a positive integer, dx(x i , y i ) is the horizontal component of the adjacent frame optical flow field of the i-th pixel point in the grayscale frame of the facial image, and dy(x i , y i ) is the vertical component of the adjacent frame optical flow field of the i-th pixel point in the grayscale frame of the facial image.

[0099] Furthermore, the method further includes:

[0100] Extract the local maximum values in the curve of the magnitude change of the optical flow field vector to obtain the set of nearest neighbor clustering center points;

[0101] Taking the set of the center points of the adjacent clusters as the starting points respectively, perform adjacent cluster analysis according to a preset amplitude difference threshold to obtain the clusters of the optical flow field vector amplitudes.

[0102] Further, the method further includes:

[0103] Traverse and calculate the mean values of the clusters of the optical flow field vector amplitudes to obtain a set of the mean values of the optical flow field vector amplitudes;

[0104] Taking the set of the mean values of the optical flow field vector amplitudes as the trend starting points respectively, perform central tendency screening on the clusters of the optical flow field vector amplitudes to obtain the screened clusters of the optical flow field vector amplitudes;

[0105] Perform mean value calculation on the screened clusters of the optical flow field vector amplitudes to obtain a set of the mean values of the screened optical flow field vector amplitudes;

[0106] Judge whether the set of the mean values of the screened optical flow field vector amplitudes is greater than or equal to a preset amplitude threshold. If so, sort the multiple facial image gray frames corresponding to the clusters of the optical flow field vector amplitudes in chronological order to obtain the micro-expression key frame sequence.

[0107] Further, the method further includes:

[0108] Collect sound signals by using a microphone arranged at the crib of the target baby to obtain a sound signal sequence;

[0109] Extract abnormal features from the sound signal sequence to obtain a set of sound abnormal features;

[0110] Based on the set of the sound abnormal features, correct the target care response plan to obtain a corrected care response plan.

[0111] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above describes specific embodiments of this specification. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0113] This specification and the accompanying drawings are merely illustrative of the present application and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications therein.

Claims

1. A gynecological baby care system, characterized in that: The system comprises: A facial image frame sequence acquisition module is used to place an infrared camera in the neonatal care room, collect facial images using the infrared camera, and obtain a facial image frame sequence; A micro-expression key frame sequence acquisition module is used to traverse the facial image frame sequence to extract micro-expression key frames and obtain a micro-expression key frame sequence; A micro-expression feature vector sequence acquisition module is used to extract spatiotemporal features of the micro-expression key frame sequence to obtain a micro-expression feature vector sequence; The target nursing response scheme acquisition module is used to match the nursing response scheme according to the micro-expression feature vector sequence to obtain the target nursing response scheme.

2. The obstetrics and gynecology baby care system according to claim 1, characterized in that: The micro-expression key frame sequence acquisition module is used to perform the following steps: Traversing the facial image frame sequence to perform grayscale processing to obtain a facial image grayscale frame sequence; Based on the Farneback optical flow method, the optical flow field vector amplitudes of adjacent frames of the facial image grayscale frame sequence are calculated to obtain an optical flow field vector amplitude change curve; Performing a nearest neighbor clustering analysis on the optical flow field vector amplitude change curve to obtain an optical flow field vector amplitude cluster; Micro-expression key frames are extracted from the facial image grayscale frame sequence based on the optical flow field vector amplitude cluster to obtain the micro-expression key frame sequence.

3. The obstetrics and gynecology baby care system according to claim 2, characterized in that: The micro-expression key frame sequence acquisition module is used to perform the following steps: Based on the Farneback optical flow method, dense optical flow calculation is performed on adjacent frames in the grayscale frame sequence of the facial image in a time-ordered order to obtain an adjacent frame optical flow field sequence, wherein each adjacent frame optical flow field in the adjacent frame optical flow field sequence is an optical flow field between a current frame and a previous frame; Traversing the adjacent frame optical flow field sequence to calculate the horizontal component and the vertical component, and obtaining an adjacent frame optical flow field horizontal component set sequence and an adjacent frame optical flow field vertical component set sequence; Using the optical flow field vector amplitude calculation function, the optical flow field horizontal component set sequence of adjacent frames and the optical flow field vertical component set sequence of adjacent frames are calculated to obtain the optical flow field vector amplitude sequence; The optical flow field vector amplitude sequence is fitted to construct the optical flow field vector amplitude change curve.

4. The obstetrics and gynecology baby care system according to claim 3, characterized in that: The optical flow field vector amplitude calculation function is: Among them, M ROI is the magnitude of the optical flow field vector, N is the number of pixels in the grayscale frame of the facial image, N is a positive integer, dx(x i ,y i ) is the horizontal component of the adjacent frame optical flow field of the i-th pixel in the grayscale frame of the facial image, dy(x i ,y i ) is the vertical component of the adjacent frame optical flow field of the i-th pixel in the grayscale frame of the facial image.

5. The obstetrics and gynecology baby care system according to claim 2, characterized in that: The micro-expression key frame sequence acquisition module is used to perform the following steps: Extracting the local maximum value in the optical flow field vector amplitude change curve to obtain a set of neighbor cluster center points; Taking the neighbor cluster center point sets as starting points, neighbor cluster analysis is performed according to a preset amplitude difference threshold to obtain the optical flow field vector amplitude cluster.

6. The obstetrics and gynecology baby care system according to claim 2, characterized in that: The micro-expression key frame sequence acquisition module is used to perform the following steps: Traversing and calculating the mean of the optical flow field vector amplitude clusters to obtain an optical flow field vector amplitude mean set; Taking the optical flow field vector amplitude mean value set as the trend starting point, performing centralized trend screening on the optical flow field vector amplitude clusters to obtain screened optical flow field vector amplitude clusters; Performing mean calculation on the filtered optical flow field vector amplitude clusters to obtain a filtered optical flow field vector amplitude mean set; It is determined whether the filtered optical flow field vector amplitude mean set is greater than or equal to a preset amplitude threshold. If so, multiple facial image grayscale frames corresponding to the optical flow field vector amplitude cluster are sorted in chronological order to obtain the micro-expression key frame sequence.

7. The obstetrics and gynecology baby care system according to claim 1, characterized in that: The target nursing response scheme acquisition module is used to perform the following steps: Using a microphone placed at the nursing bed of the target infant to collect sound signals, a sound signal sequence is obtained; Extracting abnormal features from the sound signal sequence to obtain a sound abnormal feature set; The target nursing response plan is modified based on the abnormal sound feature set to obtain a modified nursing response plan.

8. A method for caring for infants in obstetrics and gynecology, characterized in that: The method comprises: An infrared camera is installed in the neonatal care room, and facial images are collected using the infrared camera to obtain a facial image frame sequence; Traversing the facial image frame sequence to extract micro-expression key frames to obtain a micro-expression key frame sequence; Extracting spatiotemporal features of the micro-expression key frame sequence to obtain a micro-expression feature vector sequence; A nursing response plan is matched according to the micro-expression feature vector sequence to obtain a target nursing response plan.

Images