Infant lactation signal identification method and system

Abstract

The invention relates to an infant lactation signal identification method and system, and relates to the technical field of data processing, and the method comprises the steps: collecting the facial expression, limb movement and sounding features of an infant through a multi-modal sensor, and constructing a feature vector; extracting early lactation signal indexes based on the feature vectors; according to the early-stage lactation signal index, carrying out nonlinear fusion on the early-stage signal intensity, the middle-stage signal intensity and the late-stage signal intensity of infant lactation to generate a comprehensive evaluation index; dynamically calculating an early warning threshold according to the individual features of the baby and historical data; comparing the comprehensive evaluation index with an early warning threshold value to determine an early warning level; and recognizing a lactation signal of the infant according to the early warning level, and generating an intelligent nursing suggestion for the infant. According to the invention, the accuracy and timeliness of infant lactation signal identification can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, system, electronic device, and non-transitory computer-readable storage medium for recognizing infant breastfeeding signals. Background Art

[0002] Currently, identifying an infant's need for breastfeeding primarily relies on caregivers manually observing the infant's physiological and behavioral signals. Traditionally, caregivers use vision and hearing to determine the infant's needs and then breastfeed when they detect late signals (such as crying or redness). This manual observation model requires caregivers to continuously monitor the infant's condition and make informed decisions based on experience to ensure that the infant's physiological needs are met promptly.

[0003] However, manual observation relies heavily on the caregiver's focus and experience. In real-world scenarios, caregivers often have to juggle other tasks, making it easy to miss early and mid-stage nursing signals, leading to delayed responses. Furthermore, infants' hunger signals are gradual and diverse. Late crying signals often indicate a high level of hunger or anxiety, which can reduce the effectiveness of nursing and make it difficult to fully achieve the core requirement of "compliant feeding." Summary of the Invention

[0004] In response to the technical problems existing in the prior art, the present invention provides a method, system, electronic device and non-transitory computer-readable storage medium for recognizing infant breastfeeding signals, which can improve the accuracy and timeliness of infant breastfeeding signal recognition.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides a method for identifying infant breastfeeding signals, the method comprising: The facial expressions, body movements, and vocalizations of infants are collected through multimodal sensors to construct feature vectors. extracting an early lactation signal indicator based on the feature vector; According to the early breastfeeding signal indicator, nonlinearly fusing the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding to generate a comprehensive evaluation index; Dynamically calculating warning thresholds based on the individual characteristics and historical data of the infant; Determine the warning level by comparing the comprehensive evaluation index with the warning threshold; The breastfeeding signal of the infant is identified according to the warning level, and intelligent care suggestions for the infant are generated.

[0006] Optionally, the method of collecting the facial expressions, body movements, and vocalization features of the infant by a multimodal sensor and constructing a feature vector includes: Collecting facial expressions, body movements, and vocal features of an infant using a multimodal sensor, and constructing a facial feature matrix, a body movement matrix, and a vocal feature matrix of the infant respectively; A weighted summation process is performed on the facial feature matrix, the body behavior matrix and the vocal feature matrix of the infant to obtain the feature vector.

[0007] Optionally, the feature vector is expressed as: ; in, is the feature vector, F is the facial feature matrix, B is the body behavior matrix, V is the vocal feature matrix, are the first weight, the second weight, and the third weight, is the timing decay factor.

[0008] Optionally, the early breastfeeding signal intensity, mid-term signal intensity, and late breastfeeding signal intensity are nonlinearly fused according to the early breastfeeding signal indicator to generate a comprehensive evaluation index, including: obtaining an early signal intensity, a mid-term signal intensity, and a late signal intensity of the infant's breastfeeding; respectively determining logarithmic enhancement terms of the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding; The early lactation signal index is used as a benchmark, and the logarithmic enhancement item of the early signal intensity, the logarithmic enhancement item of the mid-term signal intensity, and the logarithmic enhancement item of the late signal intensity are respectively combined and weighted to obtain the comprehensive evaluation index.

[0009] Optionally, the comprehensive evaluation index is expressed as: ; in, is a comprehensive evaluation index. They are early signal intensity, mid-term signal intensity and late signal intensity, are the fourth, fifth and sixth weights respectively, They are the first signal modulation factor, the second signal modulation factor, and the third signal modulation factor respectively.

[0010] Optionally, extracting an early lactation signal indicator based on the feature vector includes: Performing weighted summation on the various early characteristic parameters of the infant based on time decay to obtain a sum of the early characteristic parameters; The early characteristic parameter and the integral operation of the characteristic vector are summed to extract the early breastfeeding signal index.

[0011] Optionally, the dynamically calculating the warning threshold based on the individual characteristics and historical data of the infant includes: Get the basic threshold for early warning; Calculating a weighted sum of various individual characteristic parameters of the infant to obtain a sum of individual characteristic parameters; Dynamically adjust the sinusoidal modulation item of the current comprehensive index representing the current breastfeeding demand of the infant and the historical maximum index to obtain a corresponding adjustment amount; The warning threshold is calculated according to the basic threshold, the individual characteristic parameter and the adjustment amount.

[0012] Optionally, determining the warning level by comparing the comprehensive evaluation index with the warning threshold includes: Calculating the difference integral between the comprehensive evaluation index and the warning threshold; Calculating the product of the difference integral and a time weight function; Calculating a derivative adjustment term of the comprehensive evaluation index over time; The warning level is determined according to the difference integral, the product and the derivative adjustment item.

[0013] Optionally, the multimodal sensor includes: an infrared camera for capturing the facial expression; an inertial measurement unit for collecting the limb movements; A high-sensitivity microphone is used to collect the vocal characteristics.

[0014] In addition, to achieve the above-mentioned purpose, the present invention also provides a baby breastfeeding signal recognition system, the system comprising: The data acquisition module is used to collect the facial expressions, body movements and vocal features of infants through multimodal sensors and construct feature vectors; a feature extraction module, configured to extract an early lactation signal indicator based on the feature vector; a comprehensive evaluation module, configured to perform nonlinear fusion of the early signal intensity, the mid-term signal intensity, and the late-term signal intensity of the infant's breastfeeding according to the early breastfeeding signal indicator to generate a comprehensive evaluation index; A threshold calculation module, configured to dynamically calculate a warning threshold based on the individual characteristics and historical data of the infant; An early warning determination module is used to determine the early warning level by comparing the comprehensive evaluation index with the early warning threshold; A signal recognition module is used to identify the infant's breastfeeding signal according to the warning level and generate intelligent care suggestions for the infant.

[0015] In addition, to achieve the above objectives, the present invention also proposes an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing a method for identifying infant breastfeeding signals as described above.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also proposes a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, it implements a method for identifying infant breastfeeding signals as described above.

[0017] The beneficial effects of the present invention are: (1) By collecting multimodal signals (including facial expressions, body movements, vocalizations, etc.), this solution can comprehensively analyze infant behavior from multiple dimensions. Each signal can provide additional clues to the breastfeeding signal, reducing the misjudgment caused by a single feature. This fusion of multidimensional features can significantly improve the recognition accuracy of infant breastfeeding signals, avoiding situations that are easily overlooked or misjudged by traditional manual observation.

[0018] (2) The present invention can extract early signal features of infants and, based on time series analysis, promptly identify infants’ breastfeeding needs. This early signal extraction and analysis means that parents or caregivers can intervene at the earliest stages of an infant’s breastfeeding needs, thereby avoiding discomfort or mood swings caused by prolonged hunger and improving the infant’s health and well-being.

[0019] (3) By dynamically adjusting the weight coefficients of each feature and using individual feature weights, the system can be personalized according to the characteristics of different infants. The infant's growth stage, health status, behavioral characteristics, etc. may affect the performance of their breastfeeding signals. This solution can adjust the model parameters in real time during the continuous clinical data optimization process, thereby ensuring that each infant's breastfeeding needs can be accurately identified and responded to.

[0020] In summary, the present invention has established an efficient, scientific, and intelligent infant breastfeeding signal recognition system through technical means such as multimodal signal acquisition, early signal recognition, multi-level signal fusion, and personalized dynamic early warning. It can accurately identify the infant's breastfeeding needs, optimize feeding timing, improve the infant's health level, improve the quality of care, and at the same time reduce human intervention errors, promoting the innovative development of infant care technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A scene diagram of a method for recognizing infant breastfeeding signals provided by the present invention; Figure 2 A flow chart of a method for identifying infant breastfeeding signals provided by the present invention; Figure 3 A schematic structural diagram of a baby breastfeeding signal recognition system provided by the present invention; Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention; Figure 5 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0024] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0025] See also Figure 1 , Figure 1 This is a scene diagram of a method for recognizing infant breastfeeding signals provided by the present invention. Figure 1 As shown, the terminal and server are connected via a network, such as a wired or wireless network. Terminals include, but are not limited to, portable devices such as mobile phones and tablets installed with various network platform applications, as well as fixed devices such as computers, kiosks, and advertising machines. The server provides various business services to users, including service push servers and user recommendation servers.

[0026] It should be noted that Figure 1 The scenario diagram of a method for identifying infant breastfeeding signals shown is only an example. The terminal, server, and application scenario described in the embodiment of the present invention are intended to more clearly illustrate the technical solution of the embodiment of the present invention, and do not constitute a limitation on the technical solution provided by the embodiment of the present invention. A person skilled in the art will appreciate that, with the evolution of the system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0027] Among them, the terminal can be used to: The facial expressions, body movements, and vocalizations of infants are collected through multimodal sensors to construct feature vectors. extracting an early lactation signal indicator based on the feature vector; According to the early breastfeeding signal indicator, nonlinearly fusing the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding to generate a comprehensive evaluation index; Dynamically calculating warning thresholds based on the individual characteristics and historical data of the infant; Determine the warning level by comparing the comprehensive evaluation index with the warning threshold; The breastfeeding signal of the infant is identified according to the warning level, and intelligent care suggestions for the infant are generated.

[0028] See also Figure 2 , provides a flow chart of a method for identifying infant breastfeeding signals of the present invention, comprising the following steps: Step 201: Collect the facial expressions, body movements and vocal features of the infant through a multimodal sensor and construct a feature vector.

[0029] Among them, the multimodal sensor includes: an infrared camera for collecting facial expressions; an inertial measurement unit for collecting limb movements; and a high-sensitivity microphone for collecting vocal characteristics.

[0030] In some embodiments, step 201 may include: Collecting facial expressions, body movements, and vocal features of an infant using a multimodal sensor, and constructing a facial feature matrix, a body movement matrix, and a vocal feature matrix of the infant respectively; A weighted summation process is performed on the facial feature matrix, the body behavior matrix and the vocal feature matrix of the infant to obtain the feature vector.

[0031] In some embodiments, the feature vector may be represented as: ; in, is the feature vector, F is the facial feature matrix, B is the body behavior matrix, V is the vocal feature matrix, are the first weight, the second weight, and the third weight, is the timing decay factor.

[0032] In specific implementation, this formula is used to fuse multimodal physiological signals, performing weighted fusion of three types of heterogeneous signals: the infant's facial expressions, limb movements, and vocal features. The influence of the time dimension is corrected by the temporal attenuation factor, and finally a comprehensive feature vector M is generated to provide quantitative input for the intelligent recognition of breastfeeding signals.

[0033] The facial feature matrix F can be obtained by capturing a sequence of infant facial images using an infrared camera or an RGB camera. Facial features can include expression parameters: mouth opening angle, sucking frequency, frown intensity, etc.; physiological parameters: facial muscle movement amplitude, skin temperature changes (such as facial congestion when crying); and temporal features: the duration and periodicity of expression changes. The matrix dimensions can be , n is the feature dimension, and t is the time window length.

[0034] The limb behavior matrix B can monitor limb movements through an inertial measurement unit (IMU) or pressure sensor. Limb behavior characteristics can include movement type: arm waving, leg kicking, head rotation, etc.; movement intensity: the three-axis composite amplitude of accelerometer and gyroscope data; behavior pattern: the continuity of movement (such as the difference between continuous kicking and intermittent movement). The matrix dimensions can be , m is the limb feature dimension.

[0035] The vocal feature matrix V can collect audio signals such as crying and swallowing sounds through a high-sensitivity microphone.

[0036] Vocalization features can include spectral features: fundamental frequency (F0), formant frequency, harmonic-to-noise ratio (HNR); time domain features: crying duration, volume fluctuation, silence interval; semantic features: crying pattern classification results extracted by deep learning models (such as LSTM). Matrix dimensions , k is the audio feature dimension.

[0037] Weight The role of is to distribute the importance of the modalities: according to the requirements of the breastfeeding signal recognition task, the contribution of the three types of signals is dynamically adjusted. For example, if the baby is in a quiet state, the facial features ( ) may dominate (such as sucking); if crying violently, the vocal characteristics ( ) weight increases.

[0038] The weights can be updated in real time through machine learning models (such as reinforcement learning), with input parameters including the baby's age (for example, newborns rely more on crying), the ambient noise level (lower when the noise is high), and the noise level of the baby (lower when the noise is high). ), historical breastfeeding behavior patterns (personalized adaptation).

[0039] It should be noted that The sum of is 1, , to avoid a single mode from completely dominating.

[0040] Used for signal timeliness correction: The physiological signals of infants have short-term memory characteristics. For example, the crying sound 10 seconds ago is less indicative of the current breastfeeding need than the crying sound at the current moment. The limb movement data that has not been updated for a long time may become invalid.

[0041] The decay rule can be expressed as: ; Where q is the decay rate coefficient (preset value, such as κ=0.2), It is the time interval between the current time and the signal acquisition time (unit: seconds).

[0042] In specific applications, if a facial feature F is collected before Δt=5 seconds, then: ; Indicates that the contribution of this feature to the current M is attenuated to 50% of the original value.

[0043] In summary, this formula synchronizes the acquisition time of F, B, and V through the time window to ensure the consistency of matrix dimensions. Adjust according to the current scene requirements , generate preliminary fusion results Multiply the attenuation factor ω to reduce the weight of historical signals and highlight the contribution of recent signals. The dimension is The eigenvector M of is used for subsequent signal analysis.

[0044] The present invention can overcome the problem that a single signal is susceptible to interference (for example, when F fails due to facial occlusion, B and V can still support judgment), and realize scene-driven feature optimization through weights and attenuation factors. The adjustment process is traceable and meets the needs of the medical and health field.

[0045] Step 202: extracting an early lactation signal indicator based on the feature vector.

[0046] In some embodiments, step 202 may include: Performing weighted summation on the various early characteristic parameters of the infant based on time decay to obtain a sum of the early characteristic parameters; The early characteristic parameter and the integral operation of the characteristic vector are summed to extract the early breastfeeding signal index.

[0047] In some embodiments, the early lactation signal indicator may be represented by: ; Among them, E is the early lactation signal indicator, It is Early characteristic parameters, is the feature importance coefficient, is the time decay rate, is the integral adjustment coefficient, is the duration of the feature, is a function of the variation of the eigenvector M over time t.

[0048] In the specific implementation, It is Early feature parameters represent the early breastfeeding signal features extracted from multimodal signals, such as facial features: sucking action frequency, mouth opening angle; limb movements: arm waving amplitude, leg pedaling frequency; vocal features: crying fundamental frequency (F0), volume fluctuation intensity.

[0049] Is the feature importance coefficient, used to adjust each feature parameter The contribution of is the time decay term, representing the characteristic parameter The timeliness of t decreases with the increase of duration t. is the decay rate coefficient.

[0050] It is a function of the change of the eigenvector M over time t, which represents the cumulative effect of the multimodal signal in the time dimension. is the integral adjustment coefficient, which is used to control the contribution ratio of the integral term to E.

[0051] Determine early feature parameters The time-dependent decay rate of The larger it is, the faster the historical signal decays. Balance the ratio of the weighted summation term to the integral term. The larger it is, the greater the influence of the integral term on E.

[0052] In some embodiments, early features related to breastfeeding needs (such as sucking movements, mild crying) can be extracted from multimodal signals (face, body, voice). For each characteristic parameter Weighted and combined with the time decay term Reduce the weight of historical signals. Integrate the feature vector M(t) over the time window to capture the cumulative effect of the signal (such as the cumulative intensity of persistent mild crying). Add the weighted sum term to the integral term to obtain the early signal indicator E.

[0053] Initial values ​​can be set based on domain knowledge (such as sucking action , fundamental frequency of crying , body movements ). Optimize through machine learning models (such as gradient descent method) The input includes historical lactation data and environmental parameters. β = 0.1-0.5 indicates that the signal decays to 37%-1% of its original value within 10 seconds. γ = 0.2-0.8 is adjusted based on the importance of the cumulative effect of the signal. This allows E to respond quickly to the onset of early lactation signals while avoiding misjudgments caused by noise accumulation.

[0054] Can be achieved through Reduce the weight of historical signals to ensure that E pays more attention to recent signals (such as movements or crying within 5 seconds) to improve the system's response speed to early breastfeeding needs. You can combine multiple early feature parameters The weighted sum of can avoid misjudgment caused by failure of a single feature (such as relying on body movements and crying when the face is occluded). Quantify the sustained strength of the signal (such as the cumulative effect of mild crying) to avoid missing reports caused by instantaneous fluctuations. Feature importance coefficient The time decay rate β can be dynamically adjusted according to the individual characteristics of the infant (such as age and breastfeeding habits) to improve personalized adaptation capabilities.

[0055] Just as an example, a baby crying slightly ( ) and accompanied by sucking movements ( ), lasting 3 seconds (t=3), β=0.2, , , The result of the weighted summation is 0.396. Assuming that the integral term M(t) is 0.8 in 3 seconds, 0.5×0.8=0.4, and the comprehensive result E=0.396+0.4=0.796.

[0056] Another example is a baby crying violently ( ) but no sucking action ( ), lasting 5 seconds (t=5), β=0.2, , , γ = 0.5. The weighted summation term is 0.133. The integral term: Assuming that the integral of M(t) in 5 seconds is 1.2, then 0.5 × 1.2 = 0.6. The comprehensive result is: E = 0.133 + 0.6 = 0.733.

[0057] It can be seen from the above examples that the present invention can effectively quantify early breastfeeding signals and provide a reliable basis for subsequent early warning and decision-making.

[0058] Step 203: Based on the early breastfeeding signal index, nonlinearly fuse the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding to generate a comprehensive evaluation index.

[0059] In some embodiments, step 203 may include: obtaining an early signal intensity, a mid-term signal intensity, and a late signal intensity of the infant's breastfeeding; respectively determining logarithmic enhancement terms of the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding; The early lactation signal index is used as a benchmark, and the logarithmic enhancement item of the early signal intensity, the logarithmic enhancement item of the mid-term signal intensity, and the logarithmic enhancement item of the late signal intensity are respectively combined and weighted to obtain the comprehensive evaluation index.

[0060] In some embodiments, the comprehensive evaluation index can be expressed as: ; in, is a comprehensive evaluation index. They are early signal intensity, mid-term signal intensity and late signal intensity, are the fourth, fifth and sixth weights respectively, They are the first signal modulation factor, the second signal modulation factor, and the third signal modulation factor respectively.

[0061] In the specific implementation, H is a comprehensive evaluation index calculated by integrating the strength of early, mid, and late signals, as well as the influencing factors of these signals. It comprehensively measures the infant's breastfeeding needs, helps the system identify the infant's breastfeeding status, and further derives feedback recommendations. As a comprehensive index, H not only considers the signal strength of each stage of the infant's life but also refines the relative importance of each stage through weighting and adjustment factors, providing a more accurate assessment result.

[0062] E is the early signal indicator calculated in the previous step. It measures the strength of the infant's initial breastfeeding need and is typically a weighted sum of multiple physiological signals, including facial expressions and body movements. E reflects the initial intensity of the infant's breastfeeding need and is a core factor in the overall comprehensive assessment process. In the formula, E, by multiplying the weighting factor and signal strength, influences the contribution of each stage of the signal to the overall assessment.

[0063] They are early signal intensity, mid-term signal intensity and late signal intensity. In some embodiments, the early signal is the best time to breastfeed, which may include: Twisting: Babies may twist their bodies to find a comfortable position.

[0064] Mouth opening: Babies may open their mouths and show their teeth in preparation for reaching for food.

[0065] Tongue sticking out: Babies will stick their tongues out to show their desire for food.

[0066] Lip licking: Babies may lick their lips, which could be due to dryness or hunger.

[0067] Head turns: Babies may turn their heads to locate food.

[0068] In some embodiments, interim signals may include: Stretching out: Your baby may stretch out a lot to try to find a position closer to your breast.

[0069] Movement begins to increase: Babies may try to grab things within reach, such as clothing or bed rails, to get attention.

[0070] Seeking movements: The baby will make more obvious seeking movements and may pull at the mother's clothing or arms.

[0071] In some embodiments, late signals may include: Crying: When a baby is already very hungry and fussy, he or she may start crying.

[0072] Limb movements: The baby may twist and turn to try to break free from the mother's arms.

[0073] Aggressive movements: Your baby's movements may become more aggressive, indicating a strong desire to escape or get attention.

[0074] Muscle tension: A baby's muscles may become very tense, especially when trying to get food.

[0075] Changes in skin color: A baby's skin may turn red from crying, which is a sign of being very hungry.

[0076] I understand. It is the signal strength of the baby in the early stages, usually representing the signal strength of the baby's needs in the early stages of breastfeeding. It is the signal strength of the baby in the middle stage, indicating the baby's demand signal in the middle stage of breastfeeding. It is the signal strength of the infant in the late stage, indicating the signal strength shown by the infant in the late stage of breastfeeding.

[0077] Signal strength Each signal represents the intensity of the baby's need for breastfeeding at different time periods. The strength of each signal reflects the degree of need at different stages of the baby's life. The system needs to combine these intensities to determine whether the baby needs to be fed or further adjust the feeding strategy.

[0078] They are used to adjust the comprehensive evaluation index of early, mid and late signals respectively The weight coefficients of the importance of the contribution. These weight coefficients determine the relative importance of the signal at each stage based on the individual characteristics of the baby or the priority set by the system. For example, if the system believes that the baby's signal in the early stage is more critical, it can be given Higher weight; if the baby's performance in the later stages is more important, you can increase The weight of .

[0079] It is to adjust the early, middle and late signal strength to the comprehensive evaluation index The modulation factors of the influence are as follows. They adjust the contribution of the signal at different stages to the final evaluation by nonlinearly adjusting the signal strength. It is mainly used to adjust the impact of the signal strength at each stage on the comprehensive evaluation index. For example, the signal at a certain stage (such as the late signal ) may be affected by different environmental factors or infant behavior patterns. , which can more accurately reflect the actual impact of late signals.

[0080] For each signal strength , through nonlinear functions The signal strength is adjusted. This logarithmic function ensures that the influence of signal strength gradually decreases as the signal strength increases, thus avoiding excessive influence of extreme values ​​on the comprehensive evaluation index.

[0081] By introducing a logarithmic adjustment function, the system can better balance the impact of signal strength on the final evaluation, especially when the signal strength is very high or very low. This nonlinear adjustment can avoid the impact of extreme fluctuations in signal strength on the evaluation results, making the system more stable and accurate.

[0082] The entire formula combines the signal strengths of the three different stages (early, middle, and late) with the early signal indicator E. Using weighting coefficients and signal modulation factors, the signals are adjusted and nonlinearly integrated to produce a comprehensive evaluation index H. This comprehensive evaluation index H reflects the strength of the infant's breastfeeding signal and can serve as the basis for subsequent early warning, feedback, and decision-making.

[0083] In summary, the present invention weights and modulates the strength of early, mid, and late signal signals, comprehensively considering the intensity of infant feeding needs at different stages and their impact on system decision-making. Through these weightings and adjustments, the system can accurately determine the intensity of the infant's feeding needs based on their performance, providing an important basis for subsequent decision-making. This nonlinear signal fusion method can effectively improve the accuracy and robustness of the system in dynamically changing environments.

[0084] Step 204: Dynamically calculate the warning threshold based on the individual characteristics and historical data of the infant.

[0085] In some embodiments, step 204 may include: Get the basic threshold for early warning; Calculating a weighted sum of various individual characteristic parameters of the infant to obtain a sum of individual characteristic parameters; Dynamically adjust the sinusoidal modulation item of the current comprehensive index representing the current breastfeeding demand of the infant and the historical maximum index to obtain a corresponding adjustment amount; The warning threshold is calculated according to the basic threshold, the individual characteristic parameter and the adjustment amount.

[0086] In some embodiments, the warning threshold may be expressed as: ; in, is the warning threshold, is the basic threshold, is the weight of the i-th individual feature, is the i-th individual characteristic, It is the largest comprehensive index in history.

[0087] In the specific implementation, It is the warning threshold used to determine whether the infant's breastfeeding needs have reached the critical value that requires intervention. This threshold depends on the individual characteristics of the infant and historical data. Through dynamic adjustment, the system can respond to the changes of the infant in real time. It is the basis for the system to make decisions. When the infant's comprehensive assessment index When the threshold is exceeded, the system will trigger the corresponding warning level and output corresponding care recommendations.

[0088] Is the basic threshold, indicating the initial setting value of the warning. It is a constant, usually set based on theoretical models or empirical data, representing the minimum detection standard for infant breastfeeding needs. Basic threshold It provides a preliminary boundary for the system, on this basis, the system will dynamically adjust the actual warning threshold T according to individual characteristics and historical data. This basic value is usually set as a reasonable constant to ensure that the warning threshold will not fall below the most basic standard.

[0089] This is used to adjust the influence of each feature in the overall judgment. Different babies may differ in various aspects, so different weights need to be assigned to each feature. It is the actual value of the individual characteristics associated with each baby, reflecting the baby's specific physiological or behavioral characteristics. Combining these weighted characteristics with other variables can help the system calculate personalized warning thresholds based on the specific situation of the baby. For example, a heavier baby may have different early signal performance, and the system can adjust the weights to to make adaptive adjustments.

[0090] It is a comprehensive evaluation index based on H and the historical maximum comprehensive index The adjustment factor is used to adjust the threshold according to the current signal strength of the baby. Here, H is the current comprehensive evaluation index, reflecting the current breastfeeding demand intensity of the baby. It is the historical maximum value, indicating the highest signal strength in the baby's history. It is usually used as a reference point to standardize the relationship between the current signal and the historical signal.

[0091] This adjustment factor uses a sine function, as and Specifically: When H approaches hour, will approach 1, causing the overall adjustment factor to approach 2, thereby increasing the threshold When H is small, much lower than When the adjustment factor is close to 1, the threshold Lower.

[0092] This adjustment mechanism allows the system to flexibly respond to the difference between the infant’s current signal strength and historical performance. For example, if the infant’s signal strength approaches its historical peak, indicating that the infant’s needs may be urgent, the system will increase the warning threshold accordingly to ensure timely intervention.

[0093] This formula dynamically adjusts the real-time warning threshold by combining basic thresholds, individual characteristics, and historical data, enabling the system to make sensitive and accurate warning decisions in different situations. This allows each baby’s warning threshold to be individually adjusted based on its characteristics, ensuring that the needs of different babies are properly identified. ,The system can dynamically adapt to the changes of the baby, ensuring that when the baby's signal reaches a historical peak, the threshold can be appropriately increased to avoid ignoring extreme signals.

[0094] Signal Strength Modulator Ensure the system can respond quickly and adjust thresholds as the infant's signals change.

[0095] This formula dynamically calculates a real-time warning threshold for infant breastfeeding signals by combining baseline thresholds, individual characteristics, historical data, and a signal strength adjustment factor. The system flexibly adjusts the threshold based on the infant's specific characteristics and historical performance, ensuring accurate and timely judgments in different situations. This provides caregivers with a scientific basis for decision-making and reduces the risk of missing or misjudging an infant's breastfeeding needs.

[0096] Step 205: Determine the warning level by comparing the comprehensive evaluation index with the warning threshold.

[0097] In some embodiments, step 205 may include: Calculating the difference integral between the comprehensive evaluation index and the warning threshold; Calculating the product of the difference integral and a time weight function; Calculating a derivative adjustment term of the comprehensive evaluation index over time; The warning level is determined according to the difference integral, the product and the derivative adjustment item.

[0098] In some embodiments, the warning level may be expressed as: ; in, is the warning level, is the time weight function, is the rate of change adjustment coefficient.

[0099] In the specific implementation, This is the warning level, reflecting the urgency of the infant's current breastfeeding needs. Based on this warning level, the system determines whether immediate intervention or appropriate measures are necessary. L is the core output of the decision-making process, directly influencing subsequent feedback and intervention measures. The warning level indicates the intensity and urgency of the infant's breastfeeding needs.

[0100] Is a comprehensive evaluation index Real-time warning threshold This difference reflects the current signal strength of the baby (via Indicates) and the warning standard set by the system (through Represents the relationship between.

[0101] When H>T, the baby's signal strength exceeds the warning threshold, which means that the baby's breastfeeding needs are more urgent and the system should issue an alarm. When H<T, the baby's needs are relatively low and the system can continue to monitor or wait. ) is a key variable used to measure the gap between an infant's current needs and the warning level. A larger gap indicates a more urgent need and a higher warning level. Conversely, a smaller gap indicates a lower warning level.

[0102] dt is a small change in time, which means the change in ( ) differences. By integrating this difference, the system can obtain the cumulative effect of the intensity of the infant's needs over time. This integral term reflects the temporal factor of the infant's needs, that is, how the intensity of the infant's needs changes over time. If the infant's needs remain above the warning threshold, the value of this integral term will gradually increase, driving the warning level up.

[0103] is a time weighting function that adjusts the contribution of time to the warning level. Specifically, as time passes, the changes in infant needs may not be linear, so a time weighting function is needed to reasonably adjust the impact of time.

[0104] For example, the baby's needs may be more urgent during certain time periods, such as late at night or early in the morning, when parents or caregivers may be slower to respond, so the time weighting of this period is may be given higher weight.

[0105] A time-weighted function adjusts the warning level calculation based on the baby's specific needs during certain time periods. Certain times, such as at night or during sleep, may be more urgent for breastfeeding, and the system can assign higher warning levels to these times based on the time of day to ensure timely intervention.

[0106] This is the rate-of-change adjustment coefficient, which is used to adjust the impact of the rate of change of the infant's comprehensive assessment index H on the warning level. That is, if the rate of change of the infant's needs is very fast, the system can appropriately amplify its contribution to the warning level.

[0107] If the infant's signal strength H changes rapidly, it means that the infant's needs may have changed dramatically, and the system should respond to these changes more sensitively. This allows the system to decide whether a quick response is needed based on how quickly the baby's needs change. If H increases rapidly in a short period of time, The impact of this change can be increased, causing the warning level to rise rapidly.

[0108] This is the rate of change of the comprehensive evaluation index H over time, indicating the rate of change of the infant's need signal over a short period of time. This allows the system to monitor the changing trend of infant needs in real time and determine the rate of increase or decrease in infant needs. It indicates the rate at which the intensity of infant needs changes over time. If the infant's need index H increases rapidly, it indicates that the infant's needs are becoming very urgent, and the system should respond immediately.

[0109] Rate of change term This indicator reflects the rate of change in the infant's breastfeeding needs. When needs change rapidly, the system responds quickly, ensuring timely intervention. This allows the system to immediately raise the alert level in the event of a sudden increase in infant feeding needs, enabling more agile decision-making.

[0110] The purpose of the entire formula is to calculate the warning level L by comprehensively evaluating multiple dimensions such as the difference between the infant's signal strength and the threshold, time factors, and the rate of change of demand. Specifically: The cumulative effect of the difference between infants' needs and warning thresholds over time was calculated. The time weight function is used to adjust the impact of infant needs on the warning level in different time periods. It is to accelerate the response to rapidly changing signals by using the rate of change of demand to ensure that the system can quickly adapt to changes in the baby's needs.

[0111] In summary, the present invention comprehensively considers multiple factors, including the intensity, timing, and rate of change of an infant's needs, to accurately calculate the infant's current warning level. Based on this warning level, the system can respond promptly, providing caregivers with a sound basis for decision-making, ensuring that the infant's needs are promptly and accurately addressed.

[0112] Step 206: Identify the infant's breastfeeding signal according to the warning level, and generate intelligent care suggestions for the infant.

[0113] In some embodiments, care recommendations may be expressed as: ; in, is the care recommendation, L is the warning level, They are the first decision weight, the second decision weight and the third decision weight respectively.

[0114] In the specific implementation, It is a care suggestion to guide the caregiver on how to take appropriate actions based on the current breastfeeding needs of the baby. This suggestion may include adjusting the feeding frequency, feeding amount, paying attention to the baby's health, etc. Care suggestion It is the output of the system, helping caregivers make timely and scientific decisions based on the baby's feeding signals and warning levels.

[0115] L represents the alert level, reflecting the urgency of the infant's current feeding need. The alert level directly influences the intensity of the care recommendations. It is a key input for generating care recommendations. The system will respond differently based on the urgency of the infant's needs. For example, if the alert level is high, the system may recommend immediate feeding; if the alert level is low, the system may recommend continued observation.

[0116] Represents the warning level Care recommendations direct impact of is the first decision weight. This item indicates that the intensity of the infant's breastfeeding needs directly affects the formulation of care recommendations.

[0117] When the warning level When it is high, it means the baby's needs are very urgent and the system will Increase the intensity of care recommendations, instructing caregivers to immediately implement necessary feeding or other interventions. Adjusted the impact of alert levels on care recommendations. In some cases, the impact of alert levels may be adjusted based on factors such as the baby's age and health.

[0118] is the cumulative effect of the warning level over time, which represents the sum of the warning levels over a period of time, where is the second decision weight.

[0119] L is the warning level calculated in real time, It is a small increment of time, and the integral term represents the total accumulation of warning levels over time.

[0120] Indicates the change or cumulative effect of the warning level over a period of time. For example, if the baby's needs remain at a high level, the integral item will increase over time, indicating that the baby's needs are becoming more and more urgent.

[0121] Weight Used to adjust the weight of this time effect in care recommendations. The introduction of time effect helps the system take into account the impact of continuous needs on care recommendations and prevents missing the best time to intervene when the baby's needs persist.

[0122] is the acceleration of the warning level, which indicates the rate of change of the warning level relative to time (i.e., acceleration). Here, is the third decision weight.

[0123] This item indicates the acceleration of the change in the warning level L. If the warning level increases sharply in a short period of time, it means that the baby's needs are quickly becoming very urgent and the system needs to respond more quickly.

[0124] Captures the speed at which the baby's needs change. If the baby's needs change dramatically in a very short period of time, it may mean that the baby's needs have reached a critical state, so the system will respond according to the acceleration item. Increase the urgency of care recommendations to ensure caregivers can respond quickly.

[0125] Weight Used to adjust the impact of this acceleration of change on care recommendations. If the baby's needs change at an accelerated rate, the system will Strengthen responses to sudden changes and ensure timely intervention.

[0126] The following factors are combined to generate care recommendations: Directly reflects the urgency of the baby's current needs; The cumulative effect of demand over time is taken into account to ensure that the system can continuously respond to continuous demand; The acceleration of demand changes is taken into account to ensure that the system can respond quickly to sudden changes.

[0127] Through the combination of these three factors, the system can provide caregivers with personalized, timely care recommendations so that appropriate feeding measures or other necessary interventions can be taken in a timely manner.

[0128] This invention provides a multi-dimensional mechanism for generating care recommendations for an infant breastfeeding signal recognition system. By combining the infant's current alert level, changes in their needs over time, and the acceleration of these changes, the system can provide caregivers with dynamically adjusted, more precise care recommendations. This approach ensures that caregivers can respond to infants' needs in a timely and informed manner, improving feeding quality and care outcomes.

[0129] See also Figure 3 , Figure 3 This is a structural diagram of a baby breastfeeding signal recognition system provided by the present invention.

[0130] like Figure 3 As shown, an embodiment of the present invention provides a system for recognizing infant breastfeeding signals, including: The data acquisition module 301 is used to collect the facial expressions, body movements and vocal features of the infant through a multimodal sensor and construct a feature vector; A feature extraction module 302 is configured to extract an early lactation signal indicator based on the feature vector; A comprehensive evaluation module 303 is configured to perform nonlinear fusion of the early signal intensity, the mid-term signal intensity, and the late-term signal intensity of the infant's breastfeeding according to the early breastfeeding signal indicator to generate a comprehensive evaluation index; A threshold calculation module 304 is used to dynamically calculate the warning threshold based on the individual characteristics and historical data of the infant; The early warning determination module 305 is used to determine the early warning level by comparing the comprehensive evaluation index with the early warning threshold; The signal recognition module 306 is configured to recognize the infant's breastfeeding signal according to the warning level and generate intelligent care suggestions for the infant.

[0131] See also Figure 4 , Figure 4 Schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented: The facial expressions, body movements, and vocalizations of infants are collected through multimodal sensors to construct feature vectors. extracting an early lactation signal indicator based on the feature vector; According to the early breastfeeding signal indicator, nonlinearly fusing the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding to generate a comprehensive evaluation index; Dynamically calculating warning thresholds based on the individual characteristics and historical data of the infant; Determine the warning level by comparing the comprehensive evaluation index with the warning threshold; The breastfeeding signal of the infant is identified according to the warning level, and intelligent care suggestions for the infant are generated.

[0132] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 500 on which a computer program 411 is stored. When the computer program 411 is executed by a processor, the following steps are implemented: The facial expressions, body movements, and vocalizations of infants are collected through multimodal sensors to construct feature vectors. extracting an early lactation signal indicator based on the feature vector; According to the early breastfeeding signal indicator, nonlinearly fusing the early signal intensity, mid-term signal intensity, and late signal intensity of the infant's breastfeeding to generate a comprehensive evaluation index; Dynamically calculating warning thresholds based on the individual characteristics and historical data of the infant; Determine the warning level by comparing the comprehensive evaluation index with the warning threshold; The breastfeeding signal of the infant is identified according to the warning level, and intelligent care suggestions for the infant are generated.

[0133] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0136] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0138] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0139] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for identifying infant breastfeeding signals, characterized in that: The method comprises: The facial expressions, body movements and vocal features of infants are collected through multimodal sensors to construct feature vectors; extracting an early lactation signal indicator based on the feature vector; According to the early breastfeeding signal index, the early signal intensity, mid-term signal intensity and late signal intensity of infant breastfeeding are nonlinearly fused to generate a comprehensive evaluation index; Dynamically calculating warning thresholds based on individual characteristics and historical data of the infant; Determine the warning level by comparing the comprehensive evaluation index with the warning threshold; The breastfeeding signal of the infant is identified according to the warning level, and intelligent care suggestions for the infant are generated.

2. The method for identifying infant breastfeeding signals according to claim 1, characterized in that: The method of collecting the facial expressions, body movements and vocalization features of the infant by means of a multimodal sensor and constructing a feature vector comprises: Collecting facial expressions, body movements and vocal features of the infant through a multimodal sensor, and constructing a facial feature matrix, a body behavior matrix and a vocal feature matrix of the infant respectively; The facial feature matrix, the body behavior matrix and the vocal feature matrix of the infant are subjected to weighted summation processing to obtain the feature vector.

3. The method for identifying infant breastfeeding signals according to claim 2, characterized in that: The feature vector is expressed as: ； in, is the feature vector, F is the facial feature matrix, B is the body behavior matrix, V is the vocal feature matrix, are the first weight, the second weight, and the third weight, respectively. is the timing decay factor.

4. The method for identifying infant breastfeeding signals according to claim 3, characterized in that: According to the early breastfeeding signal index, the early signal strength, mid-term signal strength and late signal strength of infant breastfeeding are nonlinearly fused to generate a comprehensive evaluation index, including: Obtaining the early signal intensity, mid-term signal intensity and late-term signal intensity of the infant's breastfeeding; respectively determining the logarithmic enhancement terms of the early signal intensity, the mid-term signal intensity and the late signal intensity of the infant's breastfeeding; The early lactation signal index is used as a benchmark, and the logarithmic enhancement item of the early signal strength, the logarithmic enhancement item of the mid-term signal strength, and the logarithmic enhancement item of the late signal strength are respectively combined for weighted superposition to obtain the comprehensive evaluation index.

5. The method for identifying infant breastfeeding signals according to claim 4, characterized in that: The comprehensive evaluation index is expressed as: ； in, is a comprehensive evaluation index. They are early signal intensity, mid-term signal intensity and late signal intensity. They are the fourth weight, the fifth weight and the sixth weight, They are the first signal modulation factor, the second signal modulation factor, and the third signal modulation factor respectively.

6. The method for identifying infant breastfeeding signals according to claim 5, characterized in that: The extracting of the early lactation signal indicator based on the feature vector comprises: Performing weighted summation of the various early characteristic parameters of the infant based on time attenuation to obtain a sum of the early characteristic parameters; The early characteristic parameter and the integral operation of the characteristic vector are summed to extract the early lactation signal index.

7. The method for identifying infant breastfeeding signals according to claim 6, characterized in that: The dynamically calculating the warning threshold according to the individual characteristics and historical data of the infant includes: Get the basic threshold for early warning; Calculating the weighted sum of various individual characteristic parameters of the infant to obtain a sum of individual characteristic parameters; Dynamically adjust the sinusoidal modulation item of the current comprehensive index representing the current breastfeeding demand of the infant and the historical maximum index to obtain a corresponding adjustment amount; The warning threshold is calculated according to the basic threshold, the individual characteristic parameter and the adjustment amount.

8. The method for identifying infant breastfeeding signals according to claim 7, characterized in that: The step of comparing the comprehensive evaluation index with the warning threshold to determine the warning level includes: Calculating the difference integral between the comprehensive evaluation index and the warning threshold; Calculating the product of the difference integral and a time weight function; Calculating a derivative adjustment term of the comprehensive evaluation index over time; The warning level is determined according to the difference integral, the product and the derivative adjustment term.

9. The method for identifying infant breastfeeding signals according to claim 8, characterized in that: The multimodal sensor comprises: an infrared camera for collecting the facial expressions; an inertial measurement unit for collecting the limb movements; A high-sensitivity microphone is used to collect the vocal characteristics.

10. A baby breastfeeding signal recognition system, characterized in that: The system comprises: A data acquisition module is used to collect facial expressions, body movements and vocalization features of infants through multimodal sensors to construct feature vectors; A feature extraction module, used for extracting an early lactation signal indicator based on the feature vector; A comprehensive evaluation module, for performing nonlinear fusion of the early signal strength, mid-term signal strength and late signal strength of infant breastfeeding according to the early breastfeeding signal index to generate a comprehensive evaluation index; A threshold calculation module, used to dynamically calculate the warning threshold according to the individual characteristics and historical data of the infant; An early warning determination module, used to determine the early warning level by comparing the comprehensive evaluation index with the early warning threshold; A signal recognition module is used to recognize the breastfeeding signal of the infant according to the warning level and generate intelligent care suggestions for the infant.

Images