Premature infant personalized white noise generation method and system based on multi-modal perception

Abstract

The invention relates to the technical field of premature infant pacifying, in particular to a premature infant personalized white noise generation method and system based on multi-modal perception. The method comprises the following steps: acquiring a multi-dimensional information set of the premature infant, analyzing the multi-dimensional information set of the premature infant, and determining emotional load state information and physiological pressure state information; analyzing the emotional load state information and the physiological pressure state information, and determining the real-time demand state of the premature infant; based on a preset sound material library, according to the real-time demand state of the premature infant, performing targeted combination on the sound materials to generate a white noise combination strategy; and obtaining a parent sound material set, performing sound characteristic optimization on the white noise combination strategy based on the parent sound material set according to the emotional load state and the physiological pressure state, and determining and outputting a personalized white noise signal. According to the method, the personalized white noise signals can meet the complex and dynamic nursing requirements of the premature infants, and refined and personalized pacification of the premature infants is achieved.

Description

Technical Field

[0001] This application relates to the technical field of premature infant soothing, and in particular to a method and system for generating personalized white noise for premature infants based on multi-modal perception. Background Art

[0002] Premature infants usually have relatively fragile physiological and psychological conditions. Since their nervous systems are not yet fully developed, they are more sensitive to external sound stimuli. Research shows that white noise can help premature infants relieve negative stimuli, reduce crying frequency, improve sleep quality, and promote their neural development by providing a stable acoustic environment.

[0003] However, existing methods for generating white noise for premature infants are usually based on single-dimensional observation results of premature infants, and white noise parameters are usually set statically, which is difficult to meet the complex and dynamic nursing needs of premature infants, unable to achieve refined and personalized intervention, resulting in limited individual effects of white noise. Summary of the Invention

[0004] This application provides a method and system for generating personalized white noise for premature infants based on multi-modal perception to solve the above technical problems.

[0005] In a first aspect, this application provides a method for generating personalized white noise for premature infants based on multi-modal perception, the method comprising: Obtaining a multi-dimensional information set of a premature infant, analyzing the multi-dimensional information set of the premature infant to determine emotional load state information and physiological stress state information; analyzing the emotional load state information and the physiological stress state information to determine the real-time demand state of the premature infant; based on a preset sound material library, according to the real-time demand state of the premature infant, performing targeted combination on sound materials to generate a white noise combination strategy; obtaining a maternal sound material set, and based on the maternal sound material set, optimizing the sound characteristics of the white noise combination strategy according to the emotional load state and the physiological stress state to determine and output a personalized white noise signal.

[0006] Through this solution, analyze the multi-dimensional information set of premature infants, determine the emotional load status information and physiological stress status information reflecting the emotional state and physiological state of premature infants. On this basis, starting from the dual dimensions of emotion and physiology, determine the real-time demand status of premature infants that can map the white noise needs of premature infants. Combine with the preset sound material library, specifically combine the sound materials to generate the corresponding white noise combination strategy, and further embed the maternal sound material set into the white noise combination strategy to optimize the sound characteristics of the white noise combination strategy, and provide the corresponding personalized white noise signals to the incubator for premature infants and the nursing staff respectively, so that the personalized white noise signals can meet the complex and dynamic nursing needs of premature infants, realize the refined and personalized comfort for premature infants, and significantly improve the comfort effect on premature infants.

[0007] Optionally, the multi-dimensional information set of premature infants includes cry information, facial expression information, movement information, and physiological index information. Analyzing the multi-dimensional information set of premature infants to determine the emotional load status information and physiological stress status information includes: Analyze the cry information to determine the cry sound intensity, cry frequency, and cry rhythm, and construct a cry feature vector based on this; analyze the facial expression information to determine the facial expression tension and the facial movement change rate, and construct a facial expression feature vector based on this; analyze the movement information to determine the movement amplitude and movement frequency, and construct a movement feature vector based on this; according to the cry feature vector, the facial expression feature vector, and the movement feature vector, determine the emotional load index, and use this as the emotional load status information; according to the physiological index information, extract the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability; according to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability, determine the physiological stress index, and use this as the physiological stress status information.

[0008] Through this solution, starting from two directions of emotional characteristics and physiological characteristics, by analyzing the cry information, facial expression information, and movement information, construct a cry feature vector, a facial expression feature vector, and a movement feature vector respectively, and comprehensively analyze to obtain the emotional load index reflecting the emotional load characteristics of the infant, and use this as the emotional load status information. At the same time, through the comprehensive analysis of the physiological index information, obtain the physiological stress index reflecting the physiological stress characteristics of the infant, and use this as the physiological stress status information, so as to comprehensively reflect the current emotional state and physiological stress state of premature infants, and provide a data basis for the subsequent targeted white noise analysis process.

[0009] Optionally, determining the emotional load index according to the cry feature vector, the facial expression feature vector, and the movement feature vector is specifically the following formula: ; Wherein, is the emotional load index, is the conversion coefficient of cry features, is the weight matrix of cry features, is the cry feature vector, is the conversion coefficient of expression features, is the weight matrix of expression features, is the expression feature vector, is the conversion coefficient of action features, is the weight matrix of action features, is the action feature vector.

[0010] Through this solution, by using mathematical analysis methods, according to the cry feature vector, expression feature vector and action feature vector, the influences of cry features, expression features and action features on emotion assessment are quantified respectively, and then the emotion load index is quantified, so that the emotion load index can scientifically and comprehensively reflect the current emotion load state of premature infants, and improve the accuracy of the subsequent personalized white noise analysis process.

[0011] Optionally, the determination of the physiological stress index according to the real-time heart rate, the real-time blood oxygen saturation and the real-time heart rate variability is specifically the following formula: ; where, is the physiological stress index, is the heart rate influence weight, is the real-time heart rate, is the heart rate benchmark, is the blood oxygen influence weight, is the real-time blood oxygen saturation, is the variability influence weight, is the heart rate variability.

[0012] Through this solution, by using mathematical analysis methods, according to the real-time heart rate, the real-time blood oxygen saturation and the real-time heart rate variability, the influences of heart rate, blood oxygen saturation and heart rate variability on physiological stress are quantified respectively, and then the physiological stress index is comprehensively quantified, so that the physiological stress index can scientifically and comprehensively reflect the current physiological stress state of premature infants, and further improve the accuracy of the subsequent personalized white noise analysis process.

[0013] Optionally, the analysis of the emotion load state information and the physiological stress state information to determine the real-time demand state of premature infants includes: Determine the demand state index according to the emotion load state information and the physiological stress state information, specifically the following formula: ; where, is the demand state index, is the emotional load weight, is the emotional sensitivity index, is the emotional load index, is the emotional load benchmark, physiological stress weight, is the physiological sensitivity index, is the physiological stress index, is the physiological stress benchmark, is the coupling influence parameter; according to the demand state index, determine the state mapping index, and according to the state mapping index, determine the real-time demand state of the premature infant.

[0014] Through this solution, by using mathematical analysis means, according to the emotional load state information and physiological stress state information, quantify the influence of emotional load and physiological stress on the evaluation of the premature infant's demand state, and then quantify to obtain the demand state index, so that the demand state index can accurately reflect the specific demand state of the current premature infant under the influence of emotional load and physiological stress, and further improve the accuracy and scientific nature of the subsequent personalized white noise analysis process based on the demand state.

[0015] Optionally, the determining the state mapping index according to the demand state index is specifically the following formula:

[0016] where, is the state mapping index, is the demand state index, is the first state threshold, is the second state threshold, is the third state threshold.

[0017] Through this solution, by using mathematical analysis means, according to the demand state index, map the real demand state of the premature infant to obtain the state mapping index used to represent the specific demand state of the premature infant, so as to improve the accuracy of the real-time demand state analysis process of the premature infant.

[0018] Optionally, based on the preset sound material library, according to the real-time demand state of the premature infant, a targeted combination of sound materials is performed to generate a white noise combination strategy, including: According to the real-time demand state of the premature infant, retrieve the preset sound material library to determine the sound material set corresponding to the demand state type; according to the emotional load state information and the physiological stress state information, adjust the ratio between the corresponding emotional soothing materials and physiological blood pressure lowering materials in the sound material set to determine the preselected sound material set; based on the preselected sound material set, perform a natural transition combination on each sound material in the preselected sound material set to determine the white noise combination strategy.

[0019] Through this solution, according to the real-time demand status of premature infants, the preset sound material library is retrieved to determine the sound material set corresponding to the demand status type. On this basis, according to the emotional load status information and physiological stress status information, the proportion of materials in the sound material set is adjusted to obtain a preselected sound material set, and the sound materials in the preselected sound material set are combined with natural transitions to determine the white noise combination strategy, so that the white noise combination strategy highly matches the real-time demand status of premature infants, and at the same time, the differential treatment for different demand status degrees of premature infants is reflected in the white noise combination strategy.

[0020] Optionally, the maternal sound material set includes the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal. Based on the maternal sound material set, according to the emotional load status and the physiological stress status, the sound characteristics of the white noise combination strategy are optimized to determine and output a personalized white noise signal, including: Analyze the mother's heartbeat sound signal and the mother's breathing sound signal to determine the mother's heart rate and the mother's breathing frequency; based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing frequency, determine the maternal material embedding factor; based on the white noise combination strategy, according to the maternal material embedding factor, embed the maternal sound material set into the white noise signal corresponding to the white noise combination strategy to determine and output the personalized white noise signal.

[0021] Through this solution, starting from three characteristic directions of the mother's heartbeat sound, the mother's breathing sound, and the mother's humming sound, combined with the mother's heart rate and the mother's breathing frequency obtained by analysis, the maternal material embedding factor as the embedding quantization standard of the mother's sound characteristics is determined. Then, according to the maternal material embedding factor, the maternal sound material set is embedded into the white noise signal corresponding to the white noise combination strategy to determine the personalized white noise signal, so that the white noise combination strategy is naturally embedded into the mother's sound characteristics of premature infants, further improving the soothing effect of the personalized white noise signal on premature infants.

[0022] Optionally, based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing frequency, determining the maternal material embedding factor is specifically the following formula: ; Wherein, is the maternal material embedding factor at the current time point under, is the mother's heart rate, is the heart rate embedding weight, is the mother's heartbeat sound signal, is the breathing embedding weight, is the mother's breathing frequency, is the mother's breathing sound signal, is the humming embedding weight, is the mother's humming sound signal.

[0023] Through this solution, by means of mathematical analysis, based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing frequency, the influences of the heartbeat characteristics, the breathing characteristics, and the humming characteristics on the maternal material embedding factor are quantified respectively, so as to quantitatively obtain the maternal material embedding factor, and improve the scientificity and accuracy of the maternal material embedding factor.

[0024] In a second aspect, the present application provides a preterm infant personalized white noise generation system based on multi-modal perception. The system includes: A multi-dimensional analysis module, configured to obtain a multi-dimensional information set of a preterm infant, analyze the multi-dimensional information set of the preterm infant, and determine emotional load state information and physiological stress state information; a demand analysis module, configured to analyze the emotional load state information and the physiological stress state information, and determine the real-time demand state of the preterm infant; a strategy analysis module, configured to, based on a preset sound material library, according to the real-time demand state of the preterm infant, perform targeted combination on the sound materials to generate a white noise combination strategy; an optimization output module, configured to obtain a maternal sound material set, based on the maternal sound material set, according to the emotional load state and the physiological stress state, optimize the sound characteristics of the white noise combination strategy, and determine and output a personalized white noise signal.

[0025] Optionally, the multi-dimensional analysis module is specifically configured to: Analyze the cry information to determine the cry sound intensity, the cry frequency, and the cry rhythm, and construct a cry feature vector therefrom; analyze the expression information to determine the facial expression tension and the facial movement change rate, and construct an expression feature vector therefrom; analyze the movement information to determine the movement amplitude and the movement frequency, and construct a movement feature vector therefrom; according to the cry feature vector, the expression feature vector, and the movement feature vector, determine an emotional load index, and use this as the emotional load state information; according to the physiological index information, extract the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability; according to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability, determine a physiological stress index, and use this as the physiological stress state information.

[0026] Optionally, when the multi-dimensional analysis module determines the emotional load index according to the cry feature vector, the expression feature vector, and the movement feature vector, it is specifically the following formula: ; Among them, is the emotion load index, is the cry feature conversion coefficient, is the cry feature weight matrix, is the cry feature vector, is the expression feature conversion coefficient, is the expression feature weight matrix, is the expression feature vector, is the action feature conversion coefficient, is the action feature weight matrix, is the action feature vector.

[0027] Optionally, when the multi-dimensional analysis module determines the physiological stress index according to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability, it is specifically the following formula: ; Among them, is the physiological stress index, is the heart rate influence weight, is the real-time heart rate, is the heart rate benchmark, is the blood oxygen influence weight, is the real-time blood oxygen saturation, is the variability influence weight, is the heart rate variability.

[0028] Optionally, the demand analysis module is specifically used for: Determine the demand state index according to the emotion load state information and the physiological stress state information, specifically the following formula: ; Among them, is the demand state index, is the emotion load weight, is the emotion sensitivity index, is the emotion load index, is the emotion load benchmark, physiological stress weight, is the physiological sensitivity index, is the physiological stress index, is the physiological stress benchmark, is the coupling influence parameter; determine the state mapping index according to the demand state index, and determine the real-time demand state of the premature infant according to the state mapping index.

[0029] Optionally, when determining the state mapping index according to the demand status index, the demand analysis module specifically uses the following formula:

[0030] where is the state mapping index, is the demand status index, is the first state threshold, is the second state threshold, is the third state threshold.

[0031] Optionally, the strategy analysis module is specifically configured to: Retrieve the preset sound material library according to the real-time demand status of the premature infant, and determine a sound material set corresponding to the demand status type; adjust the ratio between the corresponding emotion soothing materials and physiological blood pressure reducing materials in the sound material set according to the emotion load status information and the physiological stress status information, and determine a preselected sound material set; based on the preselected sound material set, perform a natural transition combination on each sound material in the preselected sound material set to determine the white noise combination strategy.

[0032] Optionally, the optimization output module is specifically configured to: Analyze the mother's heartbeat sound signal and the mother's breathing sound signal to determine the mother's heart rate and the mother's breathing frequency; based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, determine the maternal material embedding factor according to the mother's heart rate and the mother's breathing frequency; based on the white noise combination strategy, embed the maternal sound material set into the white noise signal corresponding to the white noise combination strategy according to the maternal material embedding factor, and determine and output the personalized white noise signal.

[0033] Optionally, when determining the maternal material embedding factor based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal according to the mother's heart rate and the mother's breathing frequency, the optimization output module specifically uses the following formula: ; where is the maternal material embedding factor at the current time point is the mother's heart rate, is the heart rate embedding weight, is the mother's heartbeat sound signal, is the breathing embedding weight, is the mother's breathing frequency, is the mother's breathing sound signal, is the mother's humming sound signal, is the humming embedding weight, is the mother's humming voice signal. Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0035] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present application; Figure 2 is a flowchart of a method for generating personalized white noise for premature infants based on multi-modal perception provided by an embodiment of the present application; Figure 3 is a schematic structural diagram of a system for generating personalized white noise for premature infants based on multi-modal perception provided by an embodiment of the present application. Detailed Embodiments

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0037] In addition, the term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0038] The following will further describe the embodiments of the present application in detail with reference to the drawings of the specification.

[0039] Existing methods for generating white noise for premature infants are usually based on the single-dimensional observation results of premature infants, and the white noise parameters are usually set statically, which are difficult to meet the complex and dynamic nursing needs of premature infants, unable to achieve refined and personalized intervention, resulting in limited individual effects of white noise.

[0040] Based on this, the present application provides a method and system for generating personalized white noise for premature infants based on multimodal perception. The multidimensional information set of premature infants is analyzed to determine the emotional load state information and physiological stress state information reflecting the emotional state and physiological state of premature infants. On this basis, from the dual dimensions of emotion and physiology, the real-time demand state of premature infants that can map the white noise demand of premature infants is determined, and the sound materials are combined in a targeted manner in combination with the preset sound material library to generate the corresponding white noise combination strategy, and the maternal sound material set is further embedded in the white noise combination strategy to achieve the optimization of the sound characteristics of the white noise combination strategy, and the corresponding personalized white noise signal is provided to the premature infant incubator and the nursing staff respectively, so that the personalized white noise signal can meet the complex and dynamic nursing needs of premature infants, realize the refinement and personalized comfort of premature infants, and significantly improve the comfort effect on premature infants.

[0041] Figure 1 A schematic diagram of an application scenario provided by the present application. In the process of using white noise to soothe premature infants, the method provided by the present application is applied so that the generated corresponding white noise signal can meet the complex and dynamic nursing needs of premature infants, and realize the refined and personalized soothing of premature infants.

[0042] Specifically, the method of the present application is applied to any server, which communicates with the premature incubator, obtains and analyzes the premature multi-dimensional information set provided by the premature incubator through the server, determines the emotional load state information and physiological stress state information reflecting the emotional state and physiological state of the premature, and on this basis, from the dual dimensions of emotion and physiology, determines the real-time demand state of the premature that can map the white noise demand of the premature, combines the sound material with the preset sound material library, generates the corresponding white noise combination strategy, and further embeds the maternal sound material set into the white noise combination strategy to optimize the sound characteristics of the white noise combination strategy, and provides the corresponding personalized white noise signal to the premature incubator and the nursing staff, so that the personalized white noise signal can meet the complex and dynamic nursing needs of the premature, realize the refinement and personalized comfort of the premature, and significantly improve the comfort effect of the premature. The specific implementation method can refer to the following embodiments.

[0043] Figure 2 This is a flowchart of a method for generating personalized white noise for premature infants based on multimodal perception provided in an embodiment of the present application. The method of this embodiment can be applied to the server in the above scenario. Figure 2 As shown, the method includes: S201. Obtain a multidimensional information set of premature infants, analyze the multidimensional information set of premature infants, and determine emotional load state information and physiological stress state information.

[0044] The multi-dimensional information set of premature infants can be multi-dimensional information that can reflect the current soothing demand state of premature infants, such as crying, expressions, movements, etc. The multi-dimensional information set of premature infants can be provided by a premature infant incubator.

[0045] The emotional load state information can be information used to characterize the current emotional load degree of premature infants.

[0046] The physiological stress state information can be information used to characterize the current physiological stress level of premature infants.

[0047] Specifically, the nervous system of premature infants is not yet fully developed and is easily affected by external stimuli, resulting in the startle reflex, causing rapid and complex changes in the emotions and physiological indicators of premature infants. White noise can provide continuous background sound, reduce the occurrence frequency of such reflexes, thereby making the physiological responses of premature infants more stable, helping premature infants better adapt to the new environment, and reducing anxiety and uneasiness. For different demand states of premature infants (such as anxiety, fatigue, stress, etc.), different white noises are needed for soothing. The different demand states of premature infants are mainly reflected in two dimensions: the emotional state and the physiological state of premature infants. Through mathematical analysis methods, starting from the two dimensions of the emotional state and the physiological state, analyze the multi-dimensional information set of premature infants to determine the emotional load state information and physiological stress state information that can reflect the current emotional state and physiological state of premature infants, providing a scientific data basis for the accurate analysis of the personalized needs of white noise for premature infants in the future.

[0048] S202. Analyze the emotional load state information and physiological stress state information to determine the real-time demand state of premature infants.

[0049] The real-time demand state of premature infants can be the corresponding specific real-time state of premature infants when white noise intervention for soothing is needed, such as anxiety, stress, etc.

[0050] Specifically, after obtaining the emotional load state information and physiological stress state information of current premature infants, since the emotional state and physiological state of premature infants both affect the white noise demand of premature infants at the same time, through mathematical analysis methods, jointly analyze the emotional load state information and physiological stress state information to obtain a quantitative index that can reflect the current demand state of premature infants, so as to map and obtain the real-time demand state of premature infants, making the subsequent screening and processing of white noise highly matched with the real-time state of premature infants.

[0051] S203. Based on the preset sound material library, according to the real-time demand state of premature infants, conduct targeted combination of sound materials to generate a white noise combination strategy.

[0052] The preset sound material library can be a preset database for storing various white noise sound materials used to soothe premature infants, and the preset sound material library can be constructed according to the opinions of premature infant care experts.

[0053] The white noise combination strategy may be a combination strategy of various types of white noises that matches the current real-time demand status of the premature infant.

[0054] Specifically, differences in the real-time demand states of premature infants (mainly differences in state types and differences in state degrees) will cause differences in the demand for white noise (white noise materials, sound intensity, frequency, etc.). According to the precise differences in state types and state degrees reflected in the real-time demand states of premature infants, the sound materials in the preset sound material library are targetedly screened, combined and processed to determine the white noise combination strategy so that it accurately matches the real-time demand states of premature infants.

[0055] S204, obtaining a mother sound material set, optimizing the sound characteristics of the white noise combination strategy based on the mother sound material set according to the emotional load state and the physiological stress state, and determining and outputting a personalized white noise signal.

[0056] The maternal sound material set may be a collection of sound materials of the mother of a premature baby, such as the mother's heartbeat, the mother's breathing, etc. The maternal sound material set may be provided by the mother of the premature baby.

[0057] The personalized white noise signal may be a highly customized white noise signal needed to soothe a currently premature infant.

[0058] Specifically, on the basis of the white noise combination strategy, embedding the mother's voice material into the white noise can evoke the baby's sense of security and intimacy. Infants and young children are particularly sensitive to the mother's voice, which can make them feel cared for and protected. The mother's voice, such as the mother's heartbeat and breathing, is a sound that the baby often hears during pregnancy. Adding the mother's voice material to the white noise can imitate the mother's physiological rhythm and help the baby relax. Therefore, based on the maternal voice material set, according to the emotional load state and the physiological stress state, the white noise combination strategy is optimized for targeted sound characteristics through mathematical analysis, so that the mother's voice characteristics are naturally integrated into the white noise, and then the corresponding personalized white noise signal is generated to improve the soothing effect of the personalized white noise signal. In the output stage of the personalized white noise signal, it needs to be output to the premature infant incubator and the nursing staff at the same time, but the output start time point needs to be staggered, first output to the nursing staff, and then output to the premature infant incubator, so that the nursing staff can monitor the personalized white noise in advance to avoid possible short irritating frequencies and improve the soothing effect of white noise.

[0059] Through this solution, analyze the multi-dimensional information set of premature infants to determine the emotional load status information and physiological stress status information reflecting the emotional state and physiological state of premature infants. On this basis, starting from the dual dimensions of emotion and physiology, determine the real-time demand status of premature infants that can map the white noise needs of premature infants. Combine with a preset sound material library, conduct targeted combination of sound materials to generate a corresponding white noise combination strategy, and further embed the maternal sound material set into the white noise combination strategy to optimize the sound characteristics of the white noise combination strategy, and provide the corresponding personalized white noise signals to the inside of the premature infant incubator and the nursing staff respectively, so that the personalized white noise signals can meet the complex and dynamic nursing needs of premature infants, achieve refined and personalized comfort for premature infants, and significantly improve the comfort effect on premature infants.

[0060] In some embodiments, analyze the crying information, determine the crying sound intensity, crying frequency, and crying rhythm, and construct a crying feature vector based on this; analyze the facial expression information, determine the facial expression tension and the facial movement change rate, and construct a facial expression feature vector based on this; analyze the movement information, determine the movement amplitude and movement frequency, and construct a movement feature vector based on this; according to the crying feature vector, facial expression feature vector, and movement feature vector, determine the emotional load index, which is used as the emotional load status information; according to the physiological index information, extract the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability; according to the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability, determine the physiological stress index, which is used as the physiological stress status information.

[0061] The multi-dimensional information set of premature infants includes crying information, facial expression information, movement information, and physiological index information.

[0062] The crying sound intensity can be the volume and loudness of the current crying of the premature infant.

[0063] The crying frequency can be the high or low pitch of the current crying sound of the premature infant.

[0064] The crying rhythm can be the time sequence pattern of the current crying of the premature infant.

[0065] The crying feature vector can be vector information used to reflect the comprehensive characteristics of the current crying of the premature infant.

[0066] The facial expression tension can be the degree of tension of the current facial muscles of the premature infant.

[0067] The facial movement change rate can be the frequency of facial expression changes of the premature infant.

[0068] The facial expression feature vector can be vector information used to reflect the comprehensive characteristics of the current facial expression of the premature infant.

[0069] The movement amplitude can be the amplitude of the current limb movement of the premature infant.

[0070] The movement frequency can be the frequency of the current limb movements of the premature infant.

[0071] The movement feature vector can be vector information used to reflect the comprehensive features of the current limb movements of the premature infant.

[0072] The emotional load index can be a quantitative value used to characterize the current emotional load intensity of the premature infant.

[0073] The real-time heart rate can be the number of heartbeats of the current premature infant per unit time (usually per minute).

[0074] The real-time blood oxygen saturation can be the degree of combination of oxygen and hemoglobin in the blood of the current premature infant.

[0075] The real-time heart rate variability can be the degree of change in the heart beat interval time of the current premature infant.

[0076] The physiological stress index can be a quantitative value used to characterize the current physiological stress intensity of the premature infant.

[0077] Specifically, infants are not yet fully mature physiologically, so their language ability and complex emotional expressions have not yet developed. Infants' emotions are usually conveyed to the outside world through crying, facial expressions, and body movements. Among them, the sound intensity, frequency, and rhythm of crying are the main quantitative indicators reflecting the characteristics of crying. Generally, the greater the sound intensity of crying, the stronger the discomfort or pain the infant feels. High-frequency crying is usually associated with urgent, anxious, or uneasy emotions. A regular crying rhythm may indicate a state of tension or excitement, while an irregular, rapid, or drawn-out rhythm may indicate discomfort or pain. Through voice recognition technology, analyze the crying information, and normalize and integrate the sound intensity, frequency, and rhythm of crying to construct a crying feature vector reflecting the comprehensive characteristics of crying. Facial expression tension and the change rate of facial movements are the main indicators reflecting the characteristics of infants' expressions. High facial expression tension usually indicates strong emotions, such as anger, fear, or pain. A high change rate of facial movements usually indicates that the infant's emotions change relatively quickly, such as from excitement to crying, showing emotional instability or strong fluctuations. Through image analysis technology, analyze the expression information, and normalize and integrate the facial expression tension and the change rate of facial movements to construct an expression feature vector reflecting the comprehensive characteristics of expressions. The amplitude and frequency of movements are the main indicators reflecting the characteristics of infants' movements. Changes in the amplitude and frequency of movements reflect changes in the intensity of the infant's emotions. Through computer vision analysis technology, analyze the infant monitoring video, track the movement trajectories of the key points of the infant's limbs, and thus determine the amplitude and frequency of movements, and normalize and integrate the amplitude and frequency of movements to construct an expression feature vector reflecting the comprehensive characteristics of movements. Through mathematical analysis methods, jointly quantitatively analyze the above-mentioned crying feature vector, expression feature vector, and movement feature vector to determine the emotional load index reflecting the current emotional load characteristics of the infant, and thus the emotional load state information.

[0078] At the same time, the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability in the physiological index information can directly map the current physiological stress level of the infant. When the infant feels stress or stress, the body will release stress hormones (such as adrenaline), resulting in an accelerated heart rate. At the same time, it will cause a change in the infant's breathing pattern, resulting in a decrease in blood oxygen saturation. Low real-time heart rate variability is related to the stress and anxiety of the infant. Through comprehensive mathematical quantitative analysis of the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability, a physiological stress index reflecting the high or low physiological stress of the infant is obtained, and this is used as the physiological stress state information.

[0079] Through this solution, starting from two directions of emotional characteristics and physiological characteristics, by analyzing cry information, expression information, and movement information, a cry feature vector, an expression feature vector, and a movement feature vector are constructed respectively. Through comprehensive analysis, an emotional load index reflecting the emotional load characteristics of the infant is obtained, which is used as the emotional load state information. At the same time, through comprehensive analysis of physiological index information, a physiological stress index reflecting the physiological stress characteristics of the infant is obtained, which is used as the physiological stress state information, so as to comprehensively reflect the current emotional state and physiological stress state of the premature infant and provide a data basis for the subsequent targeted white noise analysis process.

[0080] In some embodiments, the emotional load index is determined according to the cry feature vector, the expression feature vector, and the movement feature vector, specifically as the following formula (1): (1); Wherein, is the emotional load index, is the cry feature conversion coefficient, is the cry feature weight matrix, is the cry feature vector, is the expression feature conversion coefficient, is the expression feature weight matrix, is the expression feature vector, is the movement feature conversion coefficient, is the movement feature weight matrix, is the movement feature vector.

[0081] The cry feature conversion coefficient can be a value used to characterize the influence of cry features on the emotional load index.

[0082] The cry feature weight matrix can be a linear mapping of the cry feature vector and the weight matrix.

[0083] The expression feature conversion coefficient can be a value used to characterize the influence of expression features on the emotional load index.

[0084] The expression feature weight matrix can be a linear mapping of the expression feature vector and the weight matrix.

[0085] The movement feature conversion coefficient can be a value used to characterize the influence of movement features on the emotional load index.

[0086] The movement feature weight matrix can be a linear mapping of the movement feature vector and the weight matrix.

[0087] The cry feature conversion coefficient, the expression feature conversion coefficient, and the movement feature conversion coefficient can be optimized through regression methods (such as gradient descent, backpropagation) based on historical sample data.

[0088] The cry feature weight matrix, the expression feature weight matrix, and the motion feature weight matrix can be obtained through neural network training based on cry spectrum data, expression key point data, and motion skeleton point trajectory data.

[0089] Specifically, the pitch, volume, and frequency of the cry of premature infants often exhibit non-linear but limited characteristics. The characteristics of the cry are enhanced as the stimulation intensity and emotional state increase, but this enhancement usually does not increase infinitely but tends to saturate. The hyperbolic tangent function in formula (1) describes the contribution of cry features to the emotional load index. The hyperbolic tangent function has a natural saturation effect and can effectively handle the volatility and positive and negative laws existing in cry features; the expression features of premature infants (such as the raising of eyebrows and the trembling of the corners of the mouth) usually have the characteristics of asymmetry and exponential amplification in the impact on emotional load. In the early stage of emotional change, the changes in expressions are relatively subtle, but as the emotion gradually escalates, some strong expression changes will have a greater load impact (such as crying, facial distortion, and painful eyes). The logarithmic function in formula (1) describes the impact of the asymmetric process of expression changes from subtle to intense on the emotional load index; the motion features of premature infants (such as the lifting or irregular struggling of hands and feet) are usually a periodic motion pattern. The sine function in formula (1) describes the impact of the periodic change of motion features on the emotional load index. As a periodic function, the sine function is suitable for describing behaviors with volatility and periodic characteristics (such as regular movements of hands and feet). Considering the above impacts, the emotional load index is quantitatively obtained.

[0090] Through this solution, by using mathematical analysis means, according to the cry feature vector, the expression feature vector, and the motion feature vector, the impacts of cry features, expression features, and motion features on emotion assessment are respectively quantified, and then the emotional load index is quantitatively obtained, enabling the emotional load index to scientifically and comprehensively reflect the current emotional load state of premature infants and improving the accuracy of the subsequent personalized white noise analysis process.

[0091] In some embodiments, according to the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability, a physiological stress index is determined, specifically as the following formula (2): (2); Wherein, is the physiological stress index, is the heart rate impact weight, is the real-time heart rate, is the heart rate benchmark, is the blood oxygen impact weight, is the real-time blood oxygen saturation, is the variability impact weight, is the heart rate variability.

[0092] The heart rate impact weight can be a weight value used to characterize the degree of influence of the heart rate on physiological stress.

[0093] The heart rate benchmark can be a standard heart rate value corresponding to premature infants in the current gestational period.

[0094] The blood oxygen impact weight can be a weight value used to characterize the degree of influence of blood oxygen saturation on physiological stress.

[0095] The variability impact weight can be a weight value used to characterize the degree of influence of standard heart rate variability on physiological stress.

[0096] The heart rate impact weight, the blood oxygen impact weight, and the variability impact weight can be optimized and obtained based on historical sample data through regression methods (such as gradient descent, backpropagation).

[0097] Specifically, through in formula (2) to describe the influence of the heart rate on the physiological stress index, the hyperbolic tangent function is used to avoid non-linear bursts or excessive amplification caused by extreme heart rate values; through to describe the inverse relationship between blood oxygen saturation and the physiological stress index. When the current blood oxygen saturation is close to 100% (normal blood oxygen level), it converges to a smaller value to reflect the boundary effect of blood oxygen saturation in physiological stress modeling; through to describe the influence of heart rate variability on the physiological stress index. Since heart rate variability usually has a wide dynamic range (possibly from 10 to 100 ms), the logarithmic function helps to stabilize the data range, avoid the adverse amplification of extreme values, and more intuitively show the significant high physiological stress state when heart rate variability decreases. Considering the above influences, the physiological stress index is quantified.

[0098] Through this solution, by using mathematical analysis means, according to the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability, the influences of the heart rate, blood oxygen saturation, and heart rate variability on physiological stress are quantified respectively, and then the physiological stress index is comprehensively quantified, enabling the physiological stress index to scientifically and comprehensively reflect the physiological stress state of current premature infants, and further improving the accuracy of the subsequent personalized white noise analysis process.

[0099] In some embodiments, according to the emotional load state information and the physiological stress state information, a demand state index is determined, specifically as the following formula (3): (3); Wherein, is the demand state index, is the emotional load weight, is the emotional sensitivity index, is the emotional load index, is the emotional load benchmark, the physiological stress weight, is the physiological sensitivity index, is the physiological stress index, is the physiological stress benchmark, is the coupling influence parameter; according to the demand state index, the state mapping index is determined, and based on the state mapping index, the real-time demand state of the premature infant is determined.

[0100] The emotional load weight can be a weight value used to characterize the degree of influence of the emotional load on the demand state.

[0101] The emotional sensitivity index can be a quantitative value used to represent the sensitivity of the change in the emotional load index to the response degree of the demand state.

[0102] The emotional load benchmark can be a normal emotional load reference value.

[0103] The physiological stress weight can be a weight value used to characterize the degree of influence of the physiological stress on the demand state.

[0104] The physiological sensitivity index can be a quantitative value of the sensitivity of the change in the physiological stress index to the response degree of the demand state.

[0105] The emotional load weight, the emotional sensitivity index, the physiological stress weight, and the physiological sensitivity index can be obtained through regression analysis based on historical sample data.

[0106] The physiological stress benchmark can be a normal physiological stress reference value.

[0107] The state mapping index can be a numerical value used to map the current demand state type of the premature infant.

[0108] The coupling influence parameter can be a numerical value used to characterize the coupling influence of the emotional load and the physiological stress on the demand state. The coupling influence parameter can be obtained by converting the product between the emotional load index and the physiological stress index through a fitting coupling coefficient.

[0109] Specifically, through the Sigmoid function in formula (3) and , respectively describe the influence of the emotional load and the physiological stress on the demand state of the premature infant. The Sigmoid function is a typical S-shaped non-linear function, and its shape (close to a "parabola") allows for a gradually smooth output to the change of the input variable, which conforms to the change characteristics of emotional and physiological indicators in reality. Furthermore, the coupling influence parameter is introduced to describe the influence of the coupling between emotion and physiology on the demand state, and the demand state index is comprehensively quantified.

[0110] Through this solution, by means of mathematical analysis, based on the emotional load status information and physiological stress status information, the impacts of emotional load and physiological stress on the demand status assessment of premature infants are quantified, and then the demand status index is quantitatively obtained, enabling the demand status index to accurately reflect the specific demand status of the current premature infant under the influence of emotional load and physiological stress, thereby improving the accuracy and scientific nature of the subsequent personalized white noise analysis process based on the demand status.

[0111] In some embodiments, according to the demand status index, a state mapping index is determined, specifically as the following formula (4): (4); Wherein, is the state mapping index, is the demand status index, is the first state threshold, is the second state threshold, is the third state threshold.

[0112] The first state threshold can be the range threshold corresponding to the demand status index of premature infants in a calm state.

[0113] The second state threshold can be the range threshold corresponding to the demand status index of premature infants in an anxious state.

[0114] The third state threshold can be the range threshold corresponding to the demand status index of premature infants in a tired state.

[0115] The first state threshold, the second state threshold, and the third state threshold can be obtained through fitting analysis of historical sample data.

[0116] Specifically, after obtaining the demand status index, it is necessary to map the current real state of the premature infant according to the demand status index. There are mainly four types of real states in which premature infants have a demand for white noise: calm state (requiring background white noise in a stable state), anxious state (requiring white noise to relieve anxiety), tired state (requiring white noise to promote sleep), and stress state (requiring white noise for intervention and comfort); through formula (4), according to the numerical interval where the demand status index is located, the real demand state of the premature infant is mapped to obtain a state mapping index used to represent the specific demand state of the premature infant, thereby improving the accuracy of the real-time demand state analysis process of premature infants.

[0117] Through this solution, by means of mathematical analysis, based on the demand status index, the real demand state of the premature infant is mapped to obtain a state mapping index used to represent the specific demand state of the premature infant, thereby improving the accuracy of the real-time demand state analysis process of premature infants.

[0118] In some embodiments, according to the real-time demand state of premature infants, a preset sound material library is retrieved to determine a set of sound materials corresponding to the demand state type; according to the emotional load state information and physiological stress state information, the ratio between the corresponding emotional soothing materials and physiological blood pressure lowering materials in the set of sound materials is adjusted to determine a preselected set of sound materials; based on the preselected set of sound materials, natural transition combinations are made for each sound material in the preselected set of sound materials to determine a white noise combination strategy.

[0119] The set of sound materials can be a collection of white noise materials in the preset sound material library corresponding to the current real-time demand state of premature infants.

[0120] The emotional soothing materials can be white noise materials in the set of sound materials used to soothe the emotions of premature infants.

[0121] The physiological blood pressure lowering materials can be white noise materials in the set of sound materials used to relieve the stress of premature infants.

[0122] The preselected set of sound materials can be a collection of sound materials after adjusting the ratio between the emotional soothing materials and physiological blood pressure lowering materials in the set of sound materials.

[0123] The natural transition combination can be a process of splicing sound materials and unifying the sound intensity in the preselected set of sound materials.

[0124] Specifically, using the state mapping index corresponding to the real-time demand state of premature infants as an index, the preset sound material library is retrieved to obtain a set of sound materials corresponding to the demand state type. On this basis, due to the differences in the specific degrees of the demand states of premature infants, for example, two premature infants in the same anxious state may have different corresponding anxiety levels, and the differences in state levels bring different demands for white noise. The differences in state levels are mainly manifested in the differences in the specific emotional load and physiological stress effects endured by premature infants. Therefore, according to the differences in emotional load and physiological stress reflected in the emotional load state information and physiological stress state information, the ratio between the corresponding emotional soothing materials and physiological blood pressure lowering materials in the set of sound materials is adjusted accordingly to obtain a preselected set of sound materials. In order to reduce the abruptness of the transition between sound materials and reduce the possible stimulation of white noise to premature infants, sound material splicing and sound intensity unification processing are performed on the preselected set of sound materials to achieve natural transition combinations of each sound material and determine a white noise combination strategy.

[0125] Through this solution, according to the real-time demand status of premature infants, the preset sound material library is retrieved to determine the sound material set corresponding to the demand status type. On this basis, according to the emotional load status information and physiological stress status information, the proportion of materials in the sound material set is adjusted to obtain a preselected sound material set, and the sound materials in the preselected sound material set are combined with natural transitions to determine the white noise combination strategy, so that the white noise combination strategy highly matches the real-time demand status of premature infants, and at the same time, the differential treatment for different demand status degrees of premature infants is reflected in the white noise combination strategy.

[0126] In some embodiments, the mother's heartbeat sound signal and the mother's breathing sound signal are analyzed to determine the mother's heart rate and the mother's breathing rate; based on the mother's heartbeat sound signal, the mother's breathing sound signal and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing rate, the maternal material embedding factor is determined; based on the white noise combination strategy, according to the maternal material embedding factor, the maternal sound material set is embedded into the white noise signal corresponding to the white noise combination strategy to determine and output the personalized white noise signal.

[0127] The maternal sound material set includes the mother's heartbeat sound signal, the mother's breathing sound signal and the mother's humming sound signal.

[0128] The mother's heartbeat sound signal can be the heartbeat audio signal of the current mother of the premature infant.

[0129] The mother's breathing sound signal can be the breathing audio signal of the current mother of the premature infant.

[0130] The mother's humming sound signal can be the humming audio signal of the current mother of the premature infant.

[0131] The maternal material embedding factor can be a sound signal factor used to reflect the maternal sound embedding characteristics.

[0132] Specifically, the mother's heartbeat sound is a sound signal that premature infants have started to receive while still in the mother's body, which can quickly evoke the "sense of security" of premature infants and the memory of dependence on the mother; the mother's breathing sound is another stable and familiar sound signal during the early development stage of premature infants. The mother's breathing sound contains both rhythm and dynamic strength changes, which provides an imitation signal similar to a "breathing anchor point" for premature infants. Especially for premature infants with underdeveloped respiratory systems, this sound is beneficial for helping them learn and improve their own breathing rhythm; the mother's humming sound is one of the most active sound stimulation methods for premature infants. It carries maternal emotions and rhythm, helps relieve the anxiety of premature infants, and enhances the sense of emotional security; through mathematical analysis methods, based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing rate, through mathematical analysis methods, integrated analysis is carried out to determine the maternal material embedding factor as the embedding quantization standard of the mother's sound characteristics. Then, according to the maternal material embedding factor, the maternal sound material set is embedded into the white noise signal corresponding to the white noise combination strategy to determine the personalized white noise signal.

[0133] Through this solution, starting from three characteristic directions of the mother's heartbeat sound, the mother's breathing sound, and the mother's humming sound, combined with the mother's heart rate and the mother's breathing rate obtained through analysis, the maternal material embedding factor as the embedding quantization standard of the mother's sound characteristics is determined. Then, according to the maternal material embedding factor, the maternal sound material set is embedded into the white noise signal corresponding to the white noise combination strategy to determine the personalized white noise signal, so that the white noise combination strategy is naturally embedded with the mother's sound characteristics of premature infants, further improving the soothing effect of the personalized white noise signal on premature infants.

[0134] In some embodiments, based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing rate, the maternal material embedding factor is determined, specifically as the following formula (5): (5); Wherein, is the maternal material embedding factor at the current time point is the mother's heart rate, is the heart rate embedding weight, is the mother's heartbeat sound signal, is the breathing embedding weight, is the mother's breathing rate, is the mother's breathing sound signal, is the humming embedding weight, is the mother's humming sound signal. is the mother's humming sound signal.

[0135] The heart rate embedding weight can be a weight value used to characterize the influence relationship between the mother's heartbeat sound signal and the maternal material embedding factor.

[0136] The breathing embedding weight can be a weight value used to characterize the influence relationship between the mother's breathing sound signal and the maternal material embedding factor.

[0137] The humming embedding weight can be a weight value used to characterize the influence relationship between the mother's humming sound signal and the maternal material embedding factor.

[0138] The heart rate embedding weight, the breathing embedding weight, and the humming embedding weight can be optimized through a regression method based on the audio experimental data.

[0139] Specifically, through the in formula (5) to describe the influence of the mother's heartbeat sound and heart rate on the maternal material embedding factor. The sinusoidal function can accurately reflect the periodic changes and rhythm characteristics of the heartbeat, enabling it to change according to the physiological rhythm in the signal; through to describe the influence of the mother's breathing sound and frequency on the maternal material embedding factor. Similarly, the sinusoidal function is used to reflect the periodic changes and rhythm characteristics of breathing; through to describe the influence of the mother's humming sound on the maternal material embedding factor; and by comprehensively quantifying the above influences, the maternal material embedding factor is obtained.

[0140] Through this solution, by using mathematical analysis means, based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing frequency, the influences of the heartbeat characteristics, breathing characteristics, and humming characteristics on the maternal material embedding factor are respectively quantified, so as to quantify the maternal material embedding factor and improve the scientificity and accuracy of the maternal material embedding factor.

[0141] Figure 3 FIG. Figure 3 shows a schematic structural diagram of a personalized white noise generation system for premature infants based on multi-modal perception according to an embodiment of the present application. As

[0142] The multi-dimensional analysis module 301 is configured to obtain a multi-dimensional information set of premature infants, analyze the multi-dimensional information set of premature infants, and determine the emotional load state information and the physiological stress state information; The demand analysis module 302 is configured to analyze the emotional load state information and the physiological stress state information to determine the real-time demand state of premature infants; A strategy analysis module 303, configured to perform targeted combination of sound materials based on a preset sound material library according to the real-time demand status of the premature infant, and generate a white noise combination strategy; An optimization output module 304, configured to obtain a maternal sound material set, and optimize the sound characteristics of the white noise combination strategy based on the maternal sound material set according to the emotional load status and the physiological stress status, and determine and output a personalized white noise signal.

[0143] Optionally, the multi-dimensional analysis module 301 is specifically configured to: Analyze the cry information to determine the cry sound intensity, cry frequency, and cry rhythm, and construct a cry feature vector therewith; analyze the facial expression information to determine the facial expression tension and the facial movement change rate, and construct a facial expression feature vector therewith; analyze the movement information to determine the movement amplitude and the movement frequency, and construct a movement feature vector therewith; determine an emotional load index according to the cry feature vector, the facial expression feature vector, and the movement feature vector, and use this as the emotional load status information; extract the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability according to the physiological index information; determine a physiological stress index according to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability, and use this as the physiological stress status information.

[0144] Optionally, when the multi-dimensional analysis module 301 determines the emotional load index according to the cry feature vector, the facial expression feature vector, and the movement feature vector, it is specifically the following formula: ; Wherein, is the emotional load index, is the cry feature conversion coefficient, is the cry feature weight matrix, is the cry feature vector, is the facial expression feature conversion coefficient, is the facial expression feature weight matrix, is the facial expression feature vector, is the movement feature conversion coefficient, is the movement feature weight matrix, is the movement feature vector.

[0145] A multi-dimensional analysis module 301, configured to obtain a multi-dimensional information set of a premature infant, analyze the multi-dimensional information set of the premature infant, and determine emotional load status information and physiological stress status information; A demand analysis module 302, configured to analyze the emotional load status information and the physiological stress status information, and determine the real-time demand status of the premature infant; A strategy analysis module 303, configured to, based on a preset sound material library and according to the real-time demand state of the premature infant, perform targeted combination of sound materials to generate a white noise combination strategy; An optimization output module 304, configured to obtain a maternal sound material set, and based on the maternal sound material set, optimize the sound characteristics of the white noise combination strategy according to the emotional load state and the physiological stress state, and determine and output a personalized white noise signal.

[0146] Optionally, the multi-dimensional analysis module 301 is specifically configured to: Analyze the crying information to determine the crying sound intensity, crying sound frequency, and crying rhythm, and construct a crying feature vector therewith; analyze the facial expression information to determine the facial expression tension and the facial movement change rate, and construct a facial expression feature vector therewith; analyze the movement information to determine the movement amplitude and the movement frequency, and construct a movement feature vector therewith; according to the crying feature vector, the facial expression feature vector, and the movement feature vector, determine an emotional load index, and use this as the emotional load state information; according to the physiological index information, extract the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability; according to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability, determine a physiological stress index, and use this as the physiological stress state information.

[0147] Optionally, when the multi-dimensional analysis module 301 determines the emotional load index according to the crying feature vector, the facial expression feature vector, and the movement feature vector, it is specifically the following formula: ; Wherein, is the emotional load index, is the crying feature conversion coefficient, is the crying feature weight matrix, is the crying feature vector, is the facial expression feature conversion coefficient, is the facial expression feature weight matrix, is the facial expression feature vector, is the movement feature conversion coefficient, is the movement feature weight matrix, is the movement feature vector.

[0148] According to the emotional load state information and the physiological stress state information, determine a demand state index, specifically the following formula: ; Wherein, is the demand state index, is the emotional load weight, is the emotional sensitivity index, is the emotional load index, is the emotional load benchmark, physiological stress weight, is the physiological sensitivity index, is the physiological stress index, is the physiological stress benchmark, is the coupling influence parameter; according to the demand status index, determine the state mapping index, and according to the state mapping index, determine the real-time demand status of the premature infant.

[0149] Optionally, when the demand analysis module 302 determines the state mapping index according to the demand status index, it is specifically the following formula:

[0150] where, is the state mapping index, is the demand status index, is the first state threshold, is the second state threshold, is the third state threshold.

[0151] Optionally, the strategy analysis module 303 is specifically used for: Retrieve the preset sound material library according to the real-time demand status of the premature infant, and determine the sound material set corresponding to the demand status type; according to the emotional load status information and the physiological stress status information, adjust the ratio between the corresponding emotional soothing materials and physiological blood pressure lowering materials in the sound material set to determine the preselected sound material set; based on the preselected sound material set, perform natural transition combination on each sound material in the preselected sound material set to determine the white noise combination strategy.

[0152] Optionally, the optimization output module 304 is specifically used for: Analyze the mother's heartbeat sound signal and the mother's breathing sound signal to determine the mother's heart rate and the mother's breathing frequency; based on the mother's heartbeat sound signal, the mother's breathing sound signal and the mother's humming sound signal, determine the maternal material embedding factor according to the mother's heart rate and the mother's breathing frequency; based on the white noise combination strategy, according to the maternal material embedding factor, embed the maternal sound material set into the white noise signal corresponding to the white noise combination strategy to determine and output the personalized white noise signal.

[0153] Optionally, when determining the maternal material embedding factor based on the maternal heartbeat sound signal, the maternal breathing sound signal, and the maternal humming sound signal according to the maternal heart rate and the maternal breathing frequency, the optimization output module 304 is specifically the following formula: ; Wherein, is the maternal material embedding factor at the current time point ; is the maternal heart rate, is the heart rate embedding weight, is the maternal heartbeat sound signal, is the breathing embedding weight, is the maternal breathing frequency, is the maternal breathing sound signal, is the humming embedding weight, is the maternal humming sound signal.

[0154] The system of this embodiment can be used to execute the method of any of the above embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

Claims

1. A personalized white noise generation method for premature infants based on multimodal perception, characterized in that Including: Obtain a multi-dimensional information set of premature infants, analyze the multi-dimensional information set of premature infants, and determine emotional load status information and physiological stress status information; Analyze the emotional load status information and the physiological stress status information to determine the real-time demand status of premature infants; Based on a preset sound material library, according to the real-time demand status of premature infants, conduct a targeted combination of sound materials to generate a white noise combination strategy; Obtain a maternal sound material set, and based on the maternal sound material set, optimize the sound characteristics of the white noise combination strategy according to the emotional load status and the physiological stress status, and determine and output a personalized white noise signal.

2. The method according to claim 1, wherein The multi-dimensional information set of premature infants includes cry information, expression information, movement information, and physiological index information. Analyzing the multi-dimensional information set of premature infants to determine emotional load status information and physiological stress status information includes: Analyze the cry information to determine the cry sound intensity, cry frequency, and cry rhythm, and construct a cry feature vector based on this; Analyze the expression information to determine the facial expression tension and the facial movement change rate, and construct an expression feature vector based on this; Analyze the movement information to determine the movement amplitude and movement frequency, and construct a movement feature vector based on this; According to the cry feature vector, the expression feature vector, and the movement feature vector, determine an emotional load index, and use this as the emotional load status information; According to the physiological index information, extract the real-time heart rate, real-time blood oxygen saturation, and real-time heart rate variability; According to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability, determine a physiological stress index, and use this as the physiological stress status information.

3. The method according to claim 2, wherein The method for determining the emotional load index according to the cry feature vector, the expression feature vector, and the movement feature vector is specifically the following formula: ； Among them, is the emotional load index, is the cry feature conversion coefficient, is the cry feature weight matrix, is the cry feature vector, is the expression feature conversion coefficient, is the expression feature weight matrix, is the expression feature vector, is the action feature conversion coefficient, is the action feature weight matrix, is the action feature vector.

4. The method according to claim 2, wherein The method for determining the physiological stress index according to the real-time heart rate, the real-time blood oxygen saturation, and the real-time heart rate variability is specifically the following formula: ； Among them, is the physiological stress index, is the heart rate influence weight, is the real-time heart rate, is the heart rate benchmark, is the blood oxygen influence weight, is the real-time blood oxygen saturation, is the variability influence weight, is the heart rate variability.

5. The method according to claim 2, wherein Analyzing the emotional load status information and the physiological stress status information to determine the real-time demand status of premature infants includes: According to the emotional load status information and the physiological stress status information, determine a demand status index, specifically the following formula: ； Among them, is the demand status index, is the emotional load weight, is the emotional sensitivity index, is the emotional load index, is the emotional load benchmark, is the physiological stress weight, is the physiological sensitivity index, is the physiological stress index, is the physiological stress benchmark, is the coupling influence parameter; According to the demand status index, determine a status mapping index, and according to the status mapping index, determine the real-time demand status of the premature infants.

6. The method according to claim 5, characterized in that, The method for determining the status mapping index according to the demand status index is specifically the following formula: ; Among them, is the status mapping exponent, is the demand status exponent, is the first status threshold, is the second status threshold, is the third status threshold.

7. The method according to claim 5, characterized in that, Based on a preset sound material library, according to the real-time demand status of premature infants, conducting a targeted combination of sound materials to generate a white noise combination strategy includes: According to the real-time demand status of premature infants, retrieve the preset sound material library to determine a sound material set corresponding to the demand status type; According to the emotional load status information and the physiological stress status information, adjust the ratio between the corresponding emotional soothing materials and physiological blood pressure-lowering materials in the sound material set to determine a preselected sound material set; Based on the preselected sound material set, conduct a natural transition combination of each sound material in the preselected sound material set to determine the white noise combination strategy.

8. The method according to claim 7, wherein The maternal sound material set includes the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal. Based on the maternal sound material set, according to the emotional load state and the physiological stress state, optimize the sound characteristics of the white noise combination strategy, and determine and output a personalized white noise signal, including: Analyze the mother's heartbeat sound signal and the mother's breathing sound signal to determine the mother's heart rate and the mother's breathing frequency; Based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing frequency, determine the maternal material embedding factor; Based on the white noise combination strategy, according to the maternal material embedding factor, embed the maternal sound material set into the white noise signal corresponding to the white noise combination strategy, and determine and output the personalized white noise signal.

9. The method according to claim 8, characterized in that, The method for determining the maternal material embedding factor based on the mother's heartbeat sound signal, the mother's breathing sound signal, and the mother's humming sound signal, according to the mother's heart rate and the mother's breathing frequency, is specifically the following formula: ； Among them, is the maternal material embedding factor at the current time point for the maternal material, is the mother's heart rate, is the heart rate embedding weight, is the mother's heartbeat sound signal, is the breathing embedding weight, is the mother's breathing rate, is the mother's breathing sound signal, is the humming embedding weight, is the mother's humming sound signal.

10. A personalized white noise generation system for premature infants based on multimodal perception, characterized in that, Including: A multi-dimensional analysis module, configured to obtain a multi-dimensional information set of premature infants, analyze the multi-dimensional information set of premature infants, and determine the emotional load state information and the physiological stress state information; A demand analysis module, configured to analyze the emotional load state information and the physiological stress state information, and determine the real-time demand state of premature infants; A strategy analysis module, configured to generate a white noise combination strategy by specifically combining sound materials based on a preset sound material library according to the real-time demand state of premature infants; An optimization output module, configured to obtain a maternal sound material set, and based on the maternal sound material set, optimize the sound characteristics of the white noise combination strategy according to the emotional load state and the physiological stress state, and determine and output a personalized white noise signal.

Images