A child respiratory disease individual and group early warning method, system, device and medium based on multi-modal data

Abstract

The application discloses a kind of children respiratory disease individual and group early warning method, system, equipment and medium based on multi-modal data, it is related to medical health monitoring and artificial intelligence data processing technical field, its specific steps are as follows: collecting children basic information data and dynamic abnormal behavior data, using voiceprint recognition model and facial micro-expression recognition model to process children dynamic abnormal behavior data, construct children individualized health baseline by weighting sliding window algorithm, output frequency deviation data, construct weight distribution model to distribute weight for different abnormal behaviors, establish individual early warning unit using weighted voting algorithm and early warning, establish group early warning unit using cosine similarity algorithm, output disease aggregation area and transmission trend.The application can early identify and early warning to children respiratory abnormality, output disease aggregation area and transmission trend, can be widely applied in kindergarten, primary school, community medical care and regional health monitoring and other scenes.

Description

Technical Field

[0001] This invention relates to the field of medical and health monitoring and artificial intelligence data processing technology, and in particular to a method, system, device and medium for early warning of individual and group respiratory diseases in children based on multimodal data. Background Technology

[0002] Respiratory diseases in children, such as influenza, asthma, and bronchitis, are highly prevalent and contagious. Their early symptoms often manifest as subtle changes such as a mild cough, facial flushing, or fluctuating body temperature, making them difficult to detect in time. Especially in group settings such as kindergartens and primary schools, children's immune systems are not yet fully developed, and diseases spread rapidly. If effective monitoring and early warning are not implemented in the early stages, large-scale cluster infections can easily occur.

[0003] Current respiratory disease prevention and control mainly relies on manual temperature measurement and teacher observation records. These methods suffer from problems such as detection lag and limited coverage, failing to achieve real-time dynamic health monitoring for individuals. In school settings, traditional morning checks are static checks conducted once a day, typically only measuring temperature upon arrival at school in the morning, failing to capture dynamic symptoms that appear during breaks or in class. For example, a child may have no abnormalities during the morning check but develop frequent coughing in class in the afternoon; such dynamic symptoms would be missed. Existing smart bracelets and other devices can only collect single physiological data such as children's temperature and heart rate, failing to correlate with core features closely related to respiratory diseases, such as facial micro-expressions, resulting in insufficient warning dimensions. Existing disease warning models are mostly developed for adult data, failing to fully consider the greater fluctuations in children's physiological data. For example, children's normal body temperature is 0.3-0.5℃ higher than adults, and early symptoms of respiratory diseases are atypical; for example, early pneumonia may only manifest as an increased respiratory rate. Adult models are not suitable for children, resulting in low warning accuracy. At the same time, existing technologies lack the ability to analyze group-level data, making it impossible to analyze group characteristics such as the density of early warnings in close contact areas and the similarity of symptoms, thus making it difficult to identify the risk of disease clustering in the early stages. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is: how to achieve early individual identification and early warning of group transmission of respiratory diseases in children through multimodal data fusion and intelligent algorithm modeling, improve the accuracy of health monitoring, and reduce reliance on manual intervention and false alarm rate.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a method for early warning of individual and group respiratory diseases in children based on multimodal data, comprising, Establish a multimodal data collection system to collect dynamic abnormal behavior data and basic information data of children; Based on children's dynamic abnormal behavior data, voiceprint recognition model and facial micro-expression recognition model are used to process the children's dynamic abnormal behavior data to obtain children's abnormal behavior frequency data. Based on the frequency data of children's abnormal behaviors and the children's basic information data, a weighted sliding window algorithm is used to construct a personalized health baseline for children and output frequency deviation data. A weight allocation model is constructed and trained based on an ensemble learning algorithm to assign weights to different abnormal behaviors; Based on frequency deviation data and the weight values ​​of different abnormal behaviors, a weighted voting algorithm is used to establish individual early warning units and issue early warnings. Based on the output of individual early warning units and the cosine similarity algorithm, a group early warning unit is established to output the disease clustering area and the spread trend.

[0007] As a preferred embodiment of the method for early warning of individual and group respiratory diseases in children based on multimodal data according to the present invention, the step of establishing a multimodal data acquisition system to collect dynamic abnormal behavior data and basic information data of children includes: Establish a multimodal data collection system to collect dynamic abnormal behavior data of children; Basic information about children was obtained through a child information form provided by the school. Link children's dynamic abnormal behavior data with children's basic information data.

[0008] As a preferred embodiment of the method for early warning of individual and group respiratory diseases in children based on multimodal data according to the present invention, wherein: the step of processing the dynamic abnormal behavior data of children using a voiceprint recognition model and a facial micro-expression recognition model to obtain frequency data of abnormal behavior in children includes: Based on dynamic abnormal behavior data of children, feature extraction algorithms are used to extract the characteristics of children's abnormal behavior; Based on the aforementioned abnormal behavioral characteristics of children, a voiceprint recognition model and a facial micro-expression recognition model were established. Based on voiceprint recognition model and facial micro-expression recognition model, the system identifies children's dynamic abnormal behavior data in real time and outputs data on the frequency of children's abnormal behavior.

[0009] This invention employs a voiceprint recognition model and a facial micro-expression recognition model to automatically analyze children's speech and facial expressions during daily activities, enabling the quantitative extraction of abnormal behaviors. The voiceprint recognition model extracts acoustic features related to respiratory diseases from multi-channel speech signals and identifies the frequency of abnormal voiceprints. The facial micro-expression recognition model detects short-term muscle changes and facial expression details in children, thereby identifying abnormal facial features reflecting respiratory discomfort or fever.

[0010] As a preferred embodiment of the method for early warning of individual and group respiratory diseases in children based on multimodal data according to the present invention, the step of constructing a personalized health baseline for children through a weighted sliding window algorithm based on children's abnormal behavior frequency data and children's basic information data, and outputting frequency deviation data, includes: Based on data on the frequency of children’s abnormal behaviors and children’s basic information data, a weighted sliding window algorithm is used to construct a personalized health baseline for children. Based on the child's individualized health baseline, compare it with the child's abnormal behavior frequency data to output frequency deviation data; Establish an update mechanism to update children's personalized health baselines daily.

[0011] This invention employs a weighted sliding window algorithm to construct a personalized health baseline for each child, taking into account both the frequency of abnormal behavior data and their static individual information. By applying a sliding window to the time-series dynamic abnormal behavior data, the system can continuously track changes in a child's health status at different times and automatically adjust the window weights based on data fluctuation trends, giving more recent data greater influence.

[0012] As a preferred embodiment of the method for early warning of individual and group respiratory diseases in children based on multimodal data according to the present invention, the step of constructing and training a weight allocation model according to an ensemble learning algorithm to assign weights to different abnormal behaviors includes: Collect clinical data on pediatric respiratory diseases and construct training samples; A weight allocation model is constructed based on the random forest algorithm, and the weight allocation model is trained using training samples; Based on the trained weight allocation model, the importance of each abnormal behavior feature is calculated, expressed as follows: , in, Here, T represents the importance value of each anomalous behavior feature, T is the total number of decision trees in the random forest, and t is the t-th decision tree. This represents the cumulative decrease in the Gini coefficient of the current feature across all split nodes in the t-th tree. is the Gini coefficient of the parent node before the split, and m is the number of child nodes after the split. For the first The number of samples in each child node. Let k be the total number of samples in the parent node, and k be the total number of categories of the samples. For the first The th child node The proportion of samples of each class; Weights are assigned to different abnormal behaviors based on the importance of each abnormal behavior characteristic.

[0013] As a preferred embodiment of the method for early warning of individual and group respiratory diseases in children based on multimodal data according to the present invention, the step of establishing individual early warning units and issuing early warnings using a weighted voting algorithm based on frequency deviation data and weight values ​​of different abnormal behaviors includes: Scoring rules are developed based on frequency deviation data and the weight values ​​of different abnormal behaviors, combined with clinical thresholds. Based on the scoring rules, a weighted voting algorithm is used to calculate the individual's total risk score; Based on the individual's total risk score, risk levels are determined, and an individual early warning unit is established to output each child's individual early warning level, abnormal characteristics, and abnormal characteristic tracing information.

[0014] As a preferred embodiment of the method for early warning of individual and group respiratory diseases in children based on multimodal data according to the present invention, the step of establishing a group early warning unit by combining the output results of the individual early warning unit with the cosine similarity algorithm, and outputting the disease clustering area and transmission trend, includes: Based on the output of individual early warning units, spatial clustering units are defined; Based on the spatial clustering unit, calculate the percentage of children under warning within the spatial clustering unit; Extract the voiceprint features of all children who triggered the warning, and use the cosine similarity algorithm to determine the voiceprint clustering of children within a spatial clustering unit; Establish group early warning units and conduct group early warnings based on the proportion of children in the spatial cluster unit and the aggregation of voiceprints.

[0015] This invention aggregates and analyzes individual early warning results for each child, and introduces a cosine similarity algorithm to calculate the similarity of abnormal features between different individuals, thereby identifying potential disease clusters and transmission trends. Cosine similarity calculation measures the degree of similarity of abnormal features between different individuals. When multiple individuals exhibit high similarity in the feature space, the system groups them into the same spatial cluster unit, thus identifying potentially high-risk areas for respiratory diseases.

[0016] This invention provides an early warning system for individual and group respiratory diseases in children based on multimodal data.

[0017] To address the aforementioned technical problems, the present invention provides the following technical solution: an early warning system for individual and group respiratory diseases in children based on multimodal data, comprising: a data acquisition module, an identification module, a baseline construction module, a weight allocation module, an individual early warning module, and a group early warning module; The acquisition module establishes a multimodal data acquisition system to collect dynamic abnormal behavior data and basic information data of children. The identification module processes the children's dynamic abnormal behavior data using a voiceprint recognition model and a facial micro-expression recognition model to obtain the frequency data of the children's abnormal behavior. The baseline construction module constructs a personalized health baseline for children based on the frequency data of children's abnormal behaviors and the children's basic information data, and outputs frequency deviation data. The weight allocation module constructs and trains a weight allocation model based on an ensemble learning algorithm to assign weights to different abnormal behaviors. The individual early warning module establishes individual early warning units and issues early warnings based on frequency deviation data and the weight values ​​of different abnormal behaviors using a weighted voting algorithm. The group early warning module establishes a group early warning unit based on the output results of the individual early warning units and the cosine similarity algorithm, and outputs the disease clustering area and the spread trend.

[0018] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the aforementioned method for early warning of individual and group respiratory diseases in children based on multimodal data.

[0019] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the aforementioned method for early warning of individual and group respiratory diseases in children based on multimodal data.

[0020] The beneficial effects of this invention are as follows: This invention establishes a comprehensive perception network reflecting children's respiratory health status by integrating multi-source information such as voiceprint, facial expression, and body temperature through a multimodal data acquisition system, thus solving the problem of insufficient data dimensions in traditional monitoring methods. Through joint analysis of voiceprint recognition and facial micro-expression recognition models, the invention enables the identification and quantitative statistics of abnormal behaviors in children. By constructing personalized health baselines for children, the invention addresses the problems of false alarms and missed alarms caused by differences in the physical constitution of different children. By establishing group early warning units through a cosine similarity algorithm, the invention can identify disease clustering areas and transmission trends in a spatial dimension. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the overall process of an early warning method for individual and group respiratory diseases in children based on multimodal data, according to an embodiment of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] Example 1, referring to Figure 1 This is one embodiment of the present invention, which provides an early warning method for individual and group respiratory diseases in children based on multimodal data, including: It should be noted that with the improvement of school health management systems, early monitoring and group early warning of respiratory diseases in children have become important directions for campus health management. Children's respiratory systems are not yet fully developed, and their resistance to environmental changes and pathogen stimuli is weak. Their early symptoms often manifest as subtle changes such as mild coughing, sneezing, or facial flushing. Traditional manual observation methods are difficult to identify in a timely manner. Existing campus health monitoring relies heavily on temperature measurement or subjective observation, which has a long monitoring cycle and lacks a mechanism for continuous tracking of dynamic changes in individual health. At the same time, existing research mostly analyzes single-modal data and fails to achieve the integration and utilization of multimodal information. Traditional early warning mechanisms mostly rely on static thresholds as the basis for judgment, ignoring individual differences and behavioral patterns in children, failing to adapt to real-time changes in health status, and making it difficult to identify potential group gathering risks in the early stages.

[0025] Therefore, addressing the problems of existing monitoring methods relying on manual labor, limited data, and lack of personalized health baselines and group transmission identification capabilities, the following steps (S1-S6) are implemented: establishing a multimodal data collection system to collect dynamic abnormal behavior data and basic information data of children; processing the dynamic abnormal behavior data using voiceprint recognition and facial micro-expression recognition models; constructing personalized health baselines for children; outputting frequency deviation data; assigning weights to different abnormal behaviors; establishing individual early warning units and issuing early warnings using a weighted voting algorithm; establishing group early warning units; and outputting disease clustering areas and transmission trends. This solves the problems of existing monitoring methods relying on manual labor, limited data, and lack of personalized health baselines and group transmission identification capabilities.

[0026] S1: Establish a multimodal data collection system to collect dynamic abnormal behavior data and basic information data of children; S2: Based on the children's dynamic abnormal behavior data, the voiceprint recognition model and facial micro-expression recognition model are used to process the children's dynamic abnormal behavior data to obtain the frequency data of children's abnormal behavior. S3: Based on the frequency data of children's abnormal behaviors and the children's basic information data, a weighted sliding window algorithm is used to construct a personalized health baseline for children and output frequency deviation data. S4: Construct a weight allocation model based on the ensemble learning algorithm and train it to assign weights to different abnormal behaviors; S5: Based on the frequency deviation data and the weight values ​​of different abnormal behaviors, a weighted voting algorithm is used to establish individual early warning units and issue early warnings; S6: Based on the output of individual early warning units and the cosine similarity algorithm, establish a group early warning unit to output the disease clustering area and the spread trend.

[0027] Example 2, an embodiment of the present invention, provides a method for early warning of individual and group respiratory diseases in children based on multimodal data, based on the previous embodiment, including: In step S1, a multimodal data acquisition system is established to collect dynamic abnormal behavior data and basic information data of children, including the following steps A1-A3: A1: Establish a multimodal data collection system to collect dynamic abnormal behavior data of children.

[0028] Voiceprint data of children was collected using a planar array microphone. One planar array microphone was deployed at each of the four corners of the classroom ceiling. The microphone array spacing was set to 0.5m, the sampling rate was 48kHz, and the noise reduction level reached 40dB. The original audio signal of the classroom was collected simultaneously by the four microphones for 45 minutes, including mixed information of environmental noise such as lecturing, friction of desks and chairs, and children's coughing.

[0029] Infrared sensors were used to collect children's body temperature data. The infrared sensor is installed 1.5m above the front and back doors of the classroom. The infrared sensor works at a frequency of collecting the child's forehead temperature once every 15 minutes, and can generate 3 temperature data points in a single collection cycle of 45 minutes.

[0030] Children's facial image data is captured using a wide-angle camera: Two low-power wide-angle cameras are installed on either side of the blackboard in the classroom. The cameras have a frame rate of 15fps and a resolution of 1280×720. The wide-angle cameras work at a frequency of capturing a child's facial image once every 5 minutes, and can collect 9 original facial images of the child within a single 45-minute cycle.

[0031] A2: Obtain basic information data about children through the child information form provided by the school.

[0032] By entering the seating chart of each child's class provided by the school, and storing the coordinate information in association with the child's name, class, and other basic information, we can provide basic coordinate data for the subsequent early warning analysis layer to divide spatial clustering units and locate group risk areas.

[0033] A3: Bind children's dynamic abnormal behavior data with children's basic information data.

[0034] In step S2, based on the children's dynamic abnormal behavior data, a voiceprint recognition model and a facial micro-expression recognition model are used to process the children's dynamic abnormal behavior data to obtain the frequency data of children's abnormal behavior, including the following steps B1-B3: B1: Based on children's dynamic abnormal behavior data, feature extraction algorithms are used to extract the characteristics of children's abnormal behavior.

[0035] Extracting voiceprint features The acquired raw audio signal undergoes real-time spectral subtraction denoising to separate the ambient noise component from the target cough sound component. The separated ambient noise is then removed, and the denoised cough sound audio data is output. Voiceprint features are extracted from the denoised cough sound audio data. The MFCC feature extraction algorithm is used to slice the cough sound audio signal into 20ms / frame segments, and a Fourier transform is performed on each frame to convert the time domain to the frequency domain. Key frequency components are extracted using a Mel filter bank, and then Mel frequency cepstral coefficients are generated through discrete cosine transform. The 12-16 dimensional MFCC feature vector for each cough sound segment is output from the denoised cough sound audio data.

[0036] Extracting body temperature characteristics: An individual body temperature baseline (90% of the average body temperature) is constructed from the collected body temperature data points. Each of the three original body temperature data points is then checked to determine whether the body temperature exceeds the baseline.

[0037] Extracting facial expression features: The original facial images of children were collected and processed using the MTCNN algorithm for multi-scale face detection. First, candidate face regions were generated, followed by bounding box regression and keypoint localization. The coordinates of key feature points such as the corners of the eyes and mouth were determined for each child's face. Finally, a fixed-size facial ROI region was cropped based on the feature point coordinates, and the facial ROI region image for each child was output.

[0038] B2: Based on the aforementioned abnormal behavioral characteristics of children, establish a voiceprint recognition model and a facial micro-expression recognition model.

[0039] B3: Based on the voiceprint recognition model and facial micro-expression recognition model, identify children's dynamic abnormal behavior data in real time and output the frequency data of children's abnormal behavior.

[0040] The abnormal frequency of voiceprints is calculated. Based on the cough sound-seat ID binding result, the total number of coughs for each child within a 45-minute collection period is continuously counted, and the noise-reduced cough sound audio data of the bound seat IDs is also analyzed. The dominant frequency range of each cough sound is analyzed using voiceprint type-based dominant frequency judgment logic. If the dominant frequency is 1500Hz-2500Hz, it is determined to be a dry cough; if the dominant frequency is 500Hz-1200Hz, it is determined to be a wet cough. Finally, the combined data of cough frequency and voiceprint type for each child is output.

[0041] Calculate the abnormal temperature frequency, which is the proportion of times the individual's temperature exceeds the baseline within a single period out of the total number of 3 data collections. Output the abnormal temperature frequency data for each child in percentage form.

[0042] The frequency of facial abnormalities is calculated. When the probability of a certain type of abnormal micro-expression is ≥0.6, the child is determined to have an abnormal micro-expression during the image acquisition. Subsequently, based on the abnormal / normal judgment results of 9 acquisitions, the number of times each child exhibits at least one type of abnormal micro-expression within a single period is counted, and the frequency of facial abnormalities is output.

[0043] In this embodiment of the application, the specific steps for establishing the voiceprint recognition model and the facial micro-expression recognition model in step B2 are as follows: A voiceprint recognition model is established, and the extracted cough sound MFCC feature vector is matched with a pre-established child-specific GMM-UBM model. The similarity probability between the cough sound MFCC feature vector and the specific model is calculated. If the probability is ≥85%, the match is considered successful. The cough sound is then bound to the corresponding child's seat ID in real time, and the cough sound-seat ID binding result is output.

[0044] It should be noted that the GMM-UBM model pre-modeling process includes: collecting children's natural speech voices 10 minutes before class during the system initialization phase; inputting the speech voice MFCC features into the GMM-UBM model for training; and generating exclusive model parameters containing the Gaussian mixture distribution features of the child's voiceprint.

[0045] A facial micro-expression recognition model is established. The pixel values ​​of the facial ROI region image are normalized and then input into the MobileNetV2 model. The MobileNetV2 model extracts facial detail features through depthwise separable convolution. After feature fusion and classifier calculation, the probability values ​​of the existence of three abnormal micro-expressions, namely drooping eyelids, head tilting, and drooping corners of the mouth, are output.

[0046] It should be noted that the MobileNetV2 model training process uses 5,000 abnormal micro-expressions from children, such as drooping eyelids, head tilting, and drooping corners of the mouth, as the training sample set. By adjusting the weights of the convolutional layers and the parameters of the fully connected layers, the MobileNetV2 model is able to recognize the features of these three types of abnormal micro-expressions.

[0047] In an optional implementation, the voiceprint recognition model and facial micro-expression recognition model established in step B2 can also employ a multimodal feature recognition method based on the fusion of convolutional neural networks and recurrent neural networks. This involves performing a short-time Fourier transform on the audio data to generate a time-spectrum. Frame segmentation and keypoint detection are performed on the image data to extract facial region features. The time-spectrum is then input into a convolutional neural network to extract deep acoustic features of the speech, and the facial image sequence is input into a recurrent neural network to capture the temporal features of facial expression changes. Finally, the features from the audio and image modalities are fused to output a unified multimodal feature vector.

[0048] In another optional implementation, the voiceprint recognition model and facial micro-expression recognition model established in step B2 can also employ an abnormal behavior recognition method based on support vector machines and principal component analysis. Time-domain and frequency-domain features are extracted from the child's audio signals and facial image data, respectively. Principal component analysis is used to reduce the dimensionality of the high-dimensional feature vectors, retaining the main feature components. The dimensionality-reduced feature vectors are then input into the support vector machine classifier for model training using labeled abnormal and normal samples. In the real-time recognition stage, newly acquired speech and image features are input into the trained SVM model to determine whether they belong to the abnormal behavior category.

[0049] It should be noted that this invention, through a voiceprint recognition model, can extract feature parameters from multi-channel speech signals to identify acoustic features reflecting respiratory abnormalities. Through a facial micro-expression recognition model, it can capture short-term changes in children's facial muscles, identify subtle facial expression differences, and supplement non-acoustic abnormalities that cannot be reflected in speech signals. The combination of the voiceprint recognition model and the facial micro-expression recognition model enables the automatic identification and analysis of speech signals and facial expression changes generated by children in their daily activities, thereby enabling frequency extraction of abnormal behaviors related to respiratory diseases.

[0050] In step S3, based on the frequency data of children's abnormal behaviors and the children's basic information data, a personalized health baseline for children is constructed using a weighted sliding window algorithm, and the frequency deviation data is output, including the following steps C1-C3: C1: Based on the frequency of children's abnormal behaviors and the children's basic information data, a weighted sliding window algorithm is used to construct a personalized health baseline for children.

[0051] C2: Based on the child's personalized health baseline, compare it with the child's abnormal behavior frequency data to output frequency deviation data.

[0052] Single-cycle frequency deviation data: The relative difference between the current actual value of a child's core frequency indicators (cough frequency, abnormal body temperature frequency, abnormal facial frequency) and the corresponding baseline value within a certain 45-minute collection cycle. Single-cycle frequency deviation = (current single-cycle frequency value - personalized baseline value) / personalized baseline value.

[0053] Determine if the voiceprint type deviates from the baseline: whether the current actual voiceprint type is consistent with the voiceprint type baseline.

[0054] C3: Set up an update mechanism to update children's personalized health baselines daily.

[0055] To ensure that the baseline can dynamically adapt to changes in children's health status, the module sets up a daily after-school update mechanism. After school each day, the earliest single-cycle data (i.e., t-10 days) is removed, and the single-cycle data collected that day is added. The four types of baselines are recalculated to form a dynamically updated personalized health baseline set. The personalized baseline data for each child is output, including cough frequency baseline, individual body temperature baseline, abnormal body temperature frequency baseline, abnormal facial frequency baseline, and voiceprint type baseline.

[0056] In this embodiment of the application, the specific steps for constructing a personalized health baseline for children in step C1 are as follows: Based on the frequency of children's abnormal behaviors over 45 minutes in the past 10 school days, a weighted sliding window algorithm was used to construct a personalized health baseline, including: The baseline cough frequency is expressed as: , in, Indicates the baseline frequency of coughs. The mean cough frequency over days t-1 to t-3. The mean cough frequency over days t-4 to t-7. The mean cough frequency is t-8 to t-10 days.

[0057] The baseline body temperature for an individual is the average body temperature over a 10-day period.

[0058] The baseline for abnormal body temperature frequency is the 90th percentile of the abnormal body temperature frequency data over a 10-day period.

[0059] The baseline for facial abnormality frequency was the 90th percentile of the 10-day facial abnormality frequency data.

[0060] The voiceprint type baseline is to collect voiceprint data of children in a healthy state, distinguish whether their cough is a dry cough or a wet cough by the dominant frequency range, and mark the type as a normal voiceprint type.

[0061] In an optional implementation, step C1, which constructs a personalized health baseline for children, can also employ a dynamic health status estimation method based on Kalman filtering. This method establishes state transition equations and observation equations based on the frequency data of children's abnormal behaviors and their relationship with observational data. A covariance matrix for process noise and measurement noise is set, and an initial estimate is calculated based on historical data. The Kalman filtering prediction update mechanism is then used to correct the state estimate at each time step based on the latest dynamic abnormal behavior data. The mean and standard deviation of the long-running state estimation sequence are extracted to form the child's personalized health baseline.

[0062] In another optional implementation, step C1, which constructs a personalized health baseline for children, can also employ a baseline modeling method based on long short-term memory networks. This involves constructing a feature vector based on the frequency of abnormal behaviors and the child's basic information data, and then inputting this feature vector into an LSTM network for training. After the LSTM network converges, the latest abnormal behavior frequency data is input to predict the health behavior trend for the next time period. The predicted values ​​are compared with the actual observed values, and an exponentially weighted update mechanism is used to generate the individual health baseline for the current moment.

[0063] It should be noted that by constructing a personalized health baseline for children, more recent data can be given higher weight based on the changing trends of time-series data, thus responding to short-term health fluctuations in children and improving the timeliness and sensitivity of anomaly detection. The personalized health baseline reflects an individual's behavioral patterns and physiological characteristics under normal health conditions, serving as a benchmark for judging health deviations. When the frequency of abnormal behavior deviates significantly from the individual's baseline during a certain period, the system can calculate the frequency deviation data.

[0064] In step S4, a weight allocation model is constructed and trained according to the ensemble learning algorithm to assign weights to different abnormal behaviors, including the following steps D1-D4: D1: Collect clinical data on pediatric respiratory diseases and construct training samples.

[0065] Clinical data from 1,500 cases of pediatric respiratory diseases were collected from the pediatric departments of 3 secondary or higher hospitals and 2 primary schools in 2023-2024. There were 1,000 sick cases and 500 healthy cases. The equipment parameters were consistent with the system. Tree subsets were generated by sampling through Bootstrap, and finally 200 tree models were generated.

[0066] D2: Construct a weight allocation model based on the random forest algorithm, and train the weight allocation model using training samples.

[0067] D3: Calculate the importance of each abnormal behavior feature based on the trained weight allocation model.

[0068] D4: Assign weights to different abnormal behaviors based on the importance of each abnormal behavior characteristic.

[0069] In this embodiment of the application, the specific steps for assigning weights to different abnormal behaviors in step S4 are as follows: The weight allocation model is expressed as follows: , in, Here, T represents the importance value of each anomalous behavior feature, T is the total number of decision trees in the random forest, and t is the t-th decision tree. This represents the cumulative decrease in the Gini coefficient of the current feature across all split nodes in the t-th tree. is the Gini coefficient of the parent node before the split, and m is the number of child nodes after the split. For the first The number of samples in each child node. Let k be the total number of samples in the parent node, and k be the total number of categories of the samples. For the first The th child node The proportion of samples of each class.

[0070] Based on the importance of each abnormal behavior characteristic, weights are assigned to different abnormal behaviors: cough deviation weight 0.4, body temperature deviation weight 0.3, facial deviation weight 0.2, and voiceprint deviation weight 0.1.

[0071] In an optional implementation, the weighting of different abnormal behaviors in step S4 can also employ a feature importance weighting method based on extreme gradient boosting. The XGBoost algorithm is used to build a classification model, which is then trained using training samples. In each iteration, the residuals are fitted based on the second-order gradient information of the objective function to optimize feature splitting nodes. After the classification model training is complete, feature importance indices are calculated based on the number of times each feature is selected as a splitting node in the decision tree, its gain value, and its coverage.

[0072] In another optional implementation, the weighting of different abnormal behaviors in step S4 can also employ a Bayesian network-based feature weight inference method. This involves establishing a Bayesian network containing various abnormal behavioral features and health status variables, estimating the conditional probability distribution of each node using training samples, and calculating the probability of each feature node's influence on the target node. Mutual information is used to quantify the contribution of each abnormal behavioral feature to the disease state. The posterior weight of each feature is calculated based on the conditional probability relationship between the feature and the disease state.

[0073] It should be noted that this invention constructs a weight allocation model by introducing an ensemble learning algorithm to analyze the importance of various abnormal behavioral features in children. The weight allocation model is trained based on a large number of clinical samples and historical monitoring data, enabling it to learn the actual contribution of different abnormal behaviors to disease risk. This allows the weight allocation of different features to be generated through feature importance analysis rather than relying on subjective experience, thus accurately reflecting the correlation between various abnormal behaviors and respiratory diseases.

[0074] In step S5, based on the frequency deviation data and the weight values ​​of different abnormal behaviors, a weighted voting algorithm is used to establish individual early warning units and issue early warnings, including the following steps E1-E3: E1: Scoring rules are developed based on frequency deviation data and the weight values ​​of different abnormal behaviors, combined with clinical thresholds.

[0075] E2: Based on the scoring rules, a weighted voting algorithm is used to calculate the individual's total risk score.

[0076] The scoring rules are shown in Table 1: Table 1. Scores for Different Abnormal Behaviors

[0077] The formula for calculating the individual's total risk score is as follows: Individual risk score = 0.4 × cough score + 0.3 × body temperature score + 0.2 × facial score + 0.1 × voiceprint score.

[0078] E3: Based on the individual's total risk score, risk classification is performed, and an individual early warning unit is established to output each child's individual early warning level, abnormal characteristics, and abnormal characteristic tracing information.

[0079] Risk is categorized based on the total score: a total score < 30 points indicates low risk and no warning is required; a total score between 30 and 60 points indicates a Level 1 warning, suggesting possible early signs of disease; and a total score > 60 points indicates a Level 2 warning, suggesting a higher risk of disease.

[0080] The individual early warning unit ultimately outputs each child's individual early warning level and abnormal characteristics, as well as single-cycle traceability information (such as the current cough frequency of 3 times / cycle, which is derived from continuous counting within 45 minutes, and the collection time is the second period of the afternoon).

[0081] In step S6, based on the output of the individual early warning unit and the cosine similarity algorithm, a group early warning unit is established, which outputs the disease clustering area and the spread trend, including the following steps F1-F4: F1: Define spatial aggregation units based on the output of individual early warning units.

[0082] The defined spatial clustering unit includes a circular area with a radius of 1.5 meters, centered on the seat coordinates of each child.

[0083] It should be noted that, considering the actual situation of 0.8-1.2 meters between classroom seats, this area can cover 4-6 seats adjacent to each other in front, behind, left, and right, forming an independent spatial unit, with each unit corresponding to a close contact group.

[0084] F2: Calculate the percentage of children under warning within the spatial clustering unit based on the spatial clustering unit.

[0085] The calculation of the group proportion frequency, including the proportion of warnings within the unit and the total proportion of warnings in the class, is expressed as follows: Percentage of children in a unit that trigger Level 1 and Level 2 warnings / Total number of children in the unit × 100%.

[0086] The percentage of children in a class who triggered Level 1 and Level 2 warnings is calculated as follows: (Number of children in the class who triggered Level 1 and Level 2 warnings) / Total number of children in the class × 100%.

[0087] F3: Extract the voiceprint features of all children who triggered the warning, and use the cosine similarity algorithm to determine the voiceprint clustering of children within the spatial clustering unit.

[0088] Extract the MFCC feature parameters from the cough voiceprints of two children. Assume the MFCC feature vector of child A is... The MFCC feature vector of child B is The similarity calculation formula is as follows: , in, This represents the voiceprint similarity between two children, where u is the index of the MFCC feature vector and n is the total number of terms in the MFCC feature vector.

[0089] Extract the MFCC features of cough voiceprints of all children who triggered the warning. Calculate the voiceprint similarity between any two children who triggered the warning using the above similarity calculation formula. If the voiceprint similarity of two or more children in a spatial clustering unit is >60%, then the unit is determined to have abnormal voiceprint clustering, indicating that there may be a risk of transmission of the same type of disease within the unit.

[0090] F4: Establish a group early warning unit and conduct group early warning based on the proportion of children in the spatial cluster unit and the voiceprint clustering.

[0091] In this embodiment of the application, the specific steps for establishing the group early warning unit in step S6 are as follows: Establish a group early warning unit. If the proportion of children in the spatial cluster unit who are in the early warning unit is greater than 50%, and the voiceprint similarity of two or more children in the unit is greater than 60%, a local cluster warning is triggered, indicating that there is a risk of disease cluster transmission in the area where the unit is located. If the proportion of children in the total number of children in the class who are in the early warning unit is greater than 20%, and the proportion of children in the total number of children in the class who are in the early warning unit on the same day increases by more than 5 percentage points compared with the previous day, an overall spread warning is triggered, indicating that the disease may spread on a large scale in the class.

[0092] The group early warning unit ultimately outputs the group early warning level, coordinates of high-risk areas, and abnormal voiceprint aggregation information, providing guidance for group prevention and control in schools.

[0093] In an optional implementation, the establishment of the group early warning unit in step S6 can also employ a group clustering identification method based on a density clustering algorithm. The neighborhood radius and minimum number of sample points for the density clustering algorithm are set according to the output results of the individual early warning units. The density clustering algorithm calculates the number of sample points within the neighborhood radius of any unvisited point. When the sample point density is greater than the minimum number of sample points, the point and its neighboring points are marked as belonging to the same cluster. The algorithm iteratively executes to form multiple high-density clusters, and identifies potential disease transmission areas and outputs group early warning results based on the spatial distribution and time-series trends of each cluster.

[0094] In another optional implementation, step S6, establishing the group early warning unit, can also employ a propagation trend analysis method based on Bayesian clustering. This involves collecting the output results of individual early warning units and establishing a joint distribution model. The time and spatial dimensions are used as joint variables, and the conditional probability of disease events is calculated using a Bayesian framework. A prior distribution is introduced to model the natural distribution characteristics of the disease in time and space. Based on posterior inference, the probability of disease clustering within the current time window is calculated, and high-probability areas are considered potential disease clustering centers for early warning.

[0095] It should be noted that this invention establishes a group early warning unit based on individual early warning results. It identifies groups of individuals with similar characteristics and spatiotemporal proximity by calculating the vector similarity between abnormal features of different individuals. When multiple individuals have high similarity in the feature space, the system determines that they may be in the same disease transmission chain or potential exposure environment, thus forming a spatial clustering unit. Based on the spatial clustering unit, the system can dynamically identify disease clustering hotspots in a class and output the transmission trend by combining time-series data.

[0096] In summary, this invention constructs a personalized health baseline by combining a voiceprint recognition model and a facial micro-expression recognition model with a weighted sliding window algorithm, and integrates a learning weight allocation model and a cosine similarity algorithm. This solves the problems of traditional monitoring methods that rely on manual labor, have limited data, and lack personalized health baselines and the ability to identify group transmission, thus enabling early individual identification and early warning of group transmission of respiratory diseases in children.

[0097] Example 3, referring to Figure 1 This is one embodiment of the present invention, which provides an individual and group early warning system for pediatric respiratory diseases based on multimodal data, including: a data acquisition module, an identification module, a baseline construction module, a weight allocation module, an individual early warning module, and a group early warning module; The acquisition module establishes a multimodal data acquisition system to collect dynamic abnormal behavior data and basic information data of children. The identification module processes the children's dynamic abnormal behavior data using a voiceprint recognition model and a facial micro-expression recognition model to obtain the frequency data of the children's abnormal behavior. The baseline construction module constructs a personalized health baseline for children based on the frequency data of children's abnormal behaviors and the children's basic information data, and outputs frequency deviation data. The weight allocation module constructs and trains a weight allocation model based on an ensemble learning algorithm to assign weights to different abnormal behaviors. The individual early warning module establishes individual early warning units and issues early warnings based on frequency deviation data and the weight values ​​of different abnormal behaviors using a weighted voting algorithm. The group early warning module establishes a group early warning unit based on the output results of the individual early warning units and the cosine similarity algorithm, and outputs the disease clustering area and the spread trend.

[0098] This embodiment also provides an electronic device applicable to a method for early warning of individual and group respiratory diseases in children based on multimodal data, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for early warning of individual and group respiratory diseases in children based on multimodal data as proposed in the above embodiment.

[0099] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements an individual and group early warning method for children's respiratory diseases based on multimodal data as proposed in the above embodiments.

[0100] The storage medium proposed in this embodiment belongs to the same inventive concept as the method for implementing individual and group early warning of respiratory diseases in children based on multimodal data proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0101] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0102] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for early warning of individual and group respiratory diseases in children based on multimodal data, characterized in that: include, Establish a multimodal data collection system to collect dynamic abnormal behavior data and basic information data of children; Based on children's dynamic abnormal behavior data, voiceprint recognition model and facial micro-expression recognition model are used to process the children's dynamic abnormal behavior data to obtain children's abnormal behavior frequency data. Based on the frequency data of children's abnormal behaviors and the children's basic information data, a weighted sliding window algorithm is used to construct a personalized health baseline for children and output frequency deviation data. A weight allocation model is constructed and trained based on an ensemble learning algorithm to assign weights to different abnormal behaviors; Based on frequency deviation data and the weight values ​​of different abnormal behaviors, a weighted voting algorithm is used to establish individual early warning units and issue early warnings. Based on the output of individual early warning units and the cosine similarity algorithm, a group early warning unit is established to output the disease clustering area and the spread trend.

2. The method for early warning of individual and group respiratory diseases in children based on multimodal data as described in claim 1, characterized in that: The establishment of a multimodal data acquisition system to collect dynamic abnormal behavior data and basic information data of children includes: Establish a multimodal data collection system to collect dynamic abnormal behavior data of children; Basic information about children was obtained through a child information form provided by the school. Link children's dynamic abnormal behavior data with children's basic information data.

3. The method for early warning of individual and group respiratory diseases in children based on multimodal data as described in claim 2, characterized in that: The process involves using a voiceprint recognition model and a facial micro-expression recognition model to process children's dynamic abnormal behavior data, resulting in frequency data of children's abnormal behaviors, including: Based on dynamic abnormal behavior data of children, feature extraction algorithms are used to extract the characteristics of children's abnormal behavior; Based on the aforementioned abnormal behavioral characteristics of children, a voiceprint recognition model and a facial micro-expression recognition model were established. Based on voiceprint recognition model and facial micro-expression recognition model, the system identifies children's dynamic abnormal behavior data in real time and outputs data on the frequency of children's abnormal behavior.

4. The method for early warning of individual and group respiratory diseases in children based on multimodal data as described in claim 3, characterized in that: The process involves constructing a personalized health baseline for children based on frequency data of abnormal behaviors and children's basic information data, using a weighted sliding window algorithm, and outputting frequency deviation data, including: Based on data on the frequency of children’s abnormal behaviors and children’s basic information data, a weighted sliding window algorithm is used to construct a personalized health baseline for children. Based on the child's individualized health baseline, compare it with the child's abnormal behavior frequency data to output frequency deviation data; Establish an update mechanism to update children's personalized health baselines daily.

5. The method for early warning of individual and group respiratory diseases in children based on multimodal data as described in claim 4, characterized in that: The step of constructing and training a weight allocation model based on an ensemble learning algorithm to assign weights to different abnormal behaviors includes: Collect clinical data on pediatric respiratory diseases and construct training samples; A weight allocation model is constructed based on the random forest algorithm, and the weight allocation model is trained using training samples; Based on the trained weight allocation model, the importance of each abnormal behavior feature is calculated, expressed as follows: ， in, Here, T represents the importance value of each anomalous behavior feature, T is the total number of decision trees in the random forest, and t is the t-th decision tree. This represents the cumulative decrease in the Gini coefficient of the current feature across all split nodes in the t-th tree. is the Gini coefficient of the parent node before the split, and m is the number of child nodes after the split. For the first The number of samples in each child node. Let k be the total number of samples in the parent node, and k be the total number of categories of the samples. For the first The th child node The proportion of samples of each class; Weights are assigned to different abnormal behaviors based on the importance of each abnormal behavior characteristic.

6. The method for early warning of individual and group respiratory diseases in children based on multimodal data as described in claim 5, characterized in that: The step of establishing individual early warning units and issuing early warnings based on frequency deviation data and weight values ​​of different abnormal behaviors using a weighted voting algorithm includes: Scoring rules are developed based on frequency deviation data and the weight values ​​of different abnormal behaviors, combined with clinical thresholds. Based on the scoring rules, a weighted voting algorithm is used to calculate the individual's total risk score; Based on the individual's total risk score, risk levels are determined, and an individual early warning unit is established to output each child's individual early warning level, abnormal characteristics, and abnormal characteristic tracing information.

7. The method for early warning of individual and group respiratory diseases in children based on multimodal data as described in claim 6, characterized in that: The process involves establishing a group early warning unit based on the output of individual early warning units and a cosine similarity algorithm, outputting disease clustering areas and transmission trends, including: Based on the output of individual early warning units, spatial clustering units are defined; Based on the spatial clustering unit, calculate the percentage of children under warning within the spatial clustering unit; Extract the voiceprint features of all children who triggered the warning, and use the cosine similarity algorithm to determine the voiceprint clustering of children within a spatial clustering unit; Establish group early warning units and conduct group early warnings based on the proportion of children in the spatial cluster unit and the aggregation of voiceprints.

8. A pre-warning system for individual and group early warning of pediatric respiratory diseases based on multimodal data, employing the pre-warning method for individual and group early warning of pediatric respiratory diseases based on multimodal data as described in any one of claims 1-7, characterized in that, include: The module includes a data acquisition module, an identification module, a baseline construction module, a weight allocation module, an individual early warning module, and a group early warning module. The acquisition module establishes a multimodal data acquisition system to collect dynamic abnormal behavior data and basic information data of children. The identification module processes the children's dynamic abnormal behavior data using a voiceprint recognition model and a facial micro-expression recognition model to obtain the frequency data of the children's abnormal behavior. The baseline construction module constructs a personalized health baseline for children based on the frequency data of children's abnormal behaviors and the children's basic information data, and outputs frequency deviation data. The weight allocation module constructs and trains a weight allocation model based on an ensemble learning algorithm to assign weights to different abnormal behaviors. The individual early warning module establishes individual early warning units and issues early warnings based on frequency deviation data and the weight values ​​of different abnormal behaviors using a weighted voting algorithm. The group early warning module establishes a group early warning unit based on the output results of the individual early warning units and the cosine similarity algorithm, and outputs the disease clustering area and the spread trend.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of any one of claims 1 to 7: a method for early warning of individual and group respiratory diseases in children based on multimodal data.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of any one of claims 1 to 7, which is a method for early warning of individual and group respiratory diseases in children based on multimodal data.

Images