Fetal movement and fetal heart abnormal feature diagnosis system and method for multi-mode signal processing

Abstract

The invention provides a fetal movement and fetal heart abnormal feature diagnosis system and method for multi-mode signal processing, and relates to the technical field of health monitoring, a signal acquisition module is used for acquiring fetal heart, fetal movement and uterine contraction signals, a preprocessing module is used for denoising, a feature extraction module is used for extracting frequency and amplitude, extraction is adjusted based on all signal parameters, and the fetal movement and fetal heart abnormal feature diagnosis system is obtained. The association analysis module is used for analyzing time association, the weight distribution module is used for dynamically distributing weights, and the fusion judgment module is used for fusing features and judging abnormity. According to the invention, through multi-modal signal acquisition, dynamic adjustment feature extraction, correlation analysis and dynamic weight fusion, accurate diagnosis of fetal abnormality is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of health monitoring, in particular to a multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis system and method. BACKGROUND

[0002] With the increasing emphasis on maternal and child health, fetal monitoring technology is continuously developing, and multi-modal signals such as fetal heart, fetal movement, and uterine contraction are increasingly applied in fetal health assessment. Multi-modal fusion is a research trend because it can integrate multi-dimensional information, but existing technologies lack depth in inter-modal correlation mining, making it difficult to meet the demand for accurate diagnosis.

[0003] Traditional fetal abnormality diagnosis systems mostly use simple splicing or fixed weighting for multi-modal signal fusion, without dynamic adjustment of strategies based on signal characteristics, and without in-depth analysis of the time correlation and implicit relationship between fetal heart, fetal movement, and uterine contraction, resulting in one-sided feature extraction and fusion results that cannot truly reflect the fetal state, with poor diagnostic accuracy.

[0004] Therefore, it is necessary to design a multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis system and method to solve the problem of shallow multi-modal fusion in existing technologies, which only simply processes signals without fully mining the implicit correlation between modalities, resulting in insufficient diagnostic accuracy. SUMMARY

[0005] In view of this, the present application proposes a multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis system and method, aiming to solve the problem of insufficient diagnostic accuracy caused by simple splicing or fixed weighting of multi-modal signals in existing technologies, without mining the implicit correlation between modalities and ignoring dynamic relationships.

[0006] In one aspect, the present application proposes a multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis system, comprising:

[0007] A signal acquisition module for acquiring multi-modal signals, the multi-modal signals including fetal heart signals, fetal movement signals, and uterine contraction signals;

[0008] A preprocessing module for removing noise and interference from the multi-modal signals;

[0009] A feature extraction module for extracting key features from the multi-modal signals, the key features including frequency and amplitude;

[0010] An association analysis module for analyzing the temporal relationship of the key features of different modal signals;

[0011] A weight allocation module for dynamically allocating weights to the key features of different modal signals based on the temporal relationship of the key features of different modal signals;

[0012] fusing the key features after the dynamic allocation of weights, and determining whether the fetus is abnormal based on the fused key features;

[0013] adjusting extraction of the key features of the fetal heart signal based on a change period of the fetal heart signal and a fetal heart signal period threshold value;

[0014] adjusting extraction of the key features of the fetal movement signal based on an occurrence frequency of the fetal movement signal and a fetal movement signal frequency threshold value;

[0015] adjusting extraction of the key features based on a duration of the uterine contraction signal and a uterine contraction signal duration threshold value;

[0016] determining a correlation between the key features of different modal signals based on a minimum time interval at which the key features of different modal signals change.

[0017] Further, when adjusting extraction of the key features of the fetal heart signal based on a change period of the fetal heart signal and a fetal heart signal period threshold value, the method comprises:

[0018] when the fetal heart signal change period is greater than the fetal heart signal period threshold value, extracting frequency data of the fetal heart signal that is less than a first frequency;

[0019] when the fetal heart signal change period is less than or equal to the fetal heart signal period threshold value, extracting frequency data of the fetal heart signal that is greater than the first frequency, and a maximum amplitude value and a minimum amplitude value in each first time interval.

[0020] Further, when adjusting extraction of the key features of the fetal movement signal based on an occurrence frequency of the fetal movement signal and a fetal movement signal frequency threshold value, the method comprises:

[0021] when the fetal movement signal occurrence frequency is greater than the fetal movement signal frequency threshold value, extracting the number of occurrences of the fetal movement signal in each second time interval and an amplitude value at each occurrence;

[0022] when the fetal movement signal occurrence frequency is less than or equal to the fetal movement signal frequency threshold value, extracting an amplitude value at each occurrence of the fetal movement signal and a time interval between two occurrences of the fetal movement signal.

[0023] Further, when adjusting extraction of the key features based on a duration of the uterine contraction signal and a uterine contraction signal duration threshold value, the method comprises:

[0024] when the uterine contraction signal duration is greater than the uterine contraction signal duration threshold value, extracting an amplitude peak value of the uterine contraction signal and a time point at which the amplitude peak value occurs;

[0025] When the duration of the contraction signal is less than or equal to the duration threshold of the contraction signal, the number of occurrences of the contraction signal in every third time interval and the amplitude value at each occurrence are extracted.

[0026] Further, when determining the correlation between the key features of different modal signals based on the minimum time interval at which the key features of different modal signals change, the method comprises:

[0027] When the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude change time point of the fetal movement signal is less than the first time interval, the correlation between the frequency of the fetal heart signal and the amplitude of the fetal movement signal is a first correlation.

[0028] When the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude change time point of the fetal movement signal is greater than or equal to the first time interval, the correlation between the frequency of the fetal heart signal and the amplitude of the fetal movement signal is a second correlation.

[0029] Further, when the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude peak occurrence time point of the contraction signal or the amplitude change time point of the contraction signal is less than the second time interval, the correlation between the frequency of the fetal heart signal and the amplitude of the contraction signal or the amplitude peak of the contraction signal is a first correlation.

[0030] When the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude peak occurrence time point of the contraction signal or the amplitude change time point of the contraction signal is greater than or equal to the second time interval, the correlation between the frequency of the fetal heart signal and the amplitude of the contraction signal or the amplitude peak of the contraction signal is a second correlation.

[0031] Further, when the minimum time interval between the amplitude change time point of the fetal movement signal and the amplitude peak occurrence time point of the contraction signal or the amplitude change time point of the contraction signal is less than the third time interval, the correlation between the amplitude of the fetal movement signal and the amplitude of the contraction signal or the amplitude peak of the contraction signal is a first correlation.

[0032] When the minimum time interval between the amplitude change time point of the fetal movement signal and the amplitude peak occurrence time point of the contraction signal or the amplitude change time point of the contraction signal is greater than or equal to the third time interval, the correlation between the amplitude of the fetal movement signal and the amplitude of the contraction signal or the amplitude peak of the contraction signal is a second correlation.

[0033] Further, when dynamically assigning weights to the key features of different modal signals based on the connection of the key features of different modal signals in time, the method comprises:

[0034] multiplying the first correlation relationship weight coefficient corresponding to the first correlation relationship between the key feature of a single modal signal and the key feature of other modal signals by the number of the first correlation relationship to obtain a first correlation relationship weighted value;

[0035] multiplying the second correlation relationship weight coefficient corresponding to the second correlation relationship between the key feature of a single modal signal and the key feature of other modal signals by the number of the second correlation relationship to obtain a second correlation relationship weighted value;

[0036] adding the first correlation relationship weighted value and the second correlation relationship weighted value to obtain a correlation relationship weighted sum;

[0037] multiplying the basic weight value of the key feature of a single modal signal by the correlation relationship weighted sum to obtain a basic weight correlation weighted product;

[0038] dividing the basic weight correlation weighted product by the sum of the number of the first correlation relationship and the number of the second correlation relationship to obtain the weight value of the key feature of a single modal signal.

[0039] Further, when fusing the key features after dynamically assigning weights and judging whether the fetus is abnormal based on the fused key features, the method comprises:

[0040] multiplying the key feature value of a single modal signal by the weight value of the key feature of the corresponding single modal signal to obtain a weighted key feature value;

[0041] summing all the weighted key feature values to obtain a fusion feature value;

[0042] when the fusion feature value is greater than a fusion feature threshold value, it is determined that the fetus is abnormal;

[0043] when the fusion feature value is less than or equal to the fusion feature threshold value, it is determined that the fetus is normal.

[0044] Compared with the prior art, the beneficial effects of the present application are that the fetal movement and fetal heart abnormality feature diagnosis system for multi-modal signal processing of the present application can dynamically adjust the extraction strategy according to each signal feature, deeply excavate the implicit relationship between modalities, improve the fusion accuracy through dynamic weight distribution, and more accurately judge whether the fetus is abnormal, thereby effectively overcoming the defect of the prior art that the multi-modal fusion is shallow.

[0045] In another aspect, the present application provides a multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis method, comprising:

[0046] S100, collecting multi-modal signals, the multi-modal signals comprising: fetal heart signals, fetal movement signals, and uterine contraction signals;

[0047] S200, removing noise and interference from the multi-modal signals;

[0048] S300, extracting key features from the multi-modal signals, the key features comprising: frequency and amplitude;

[0049] S400, analyzing the temporal relationship of the key features of different modal signals;

[0050] S500, dynamically assigning weights to the key features of different modal signals based on the temporal relationship of the key features of different modal signals;

[0051] S600, fusing the key features after dynamically assigning weights, and determining whether the fetus is abnormal based on the fused key features.

[0052] It can be understood that the above-mentioned multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis method and system have the same beneficial effects, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0053] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the present application. Moreover, the same reference numerals are used throughout the various drawings to designate identical parts. In the drawings:

[0054] Figure 1 The multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis system function block diagram provided for the embodiments of the present application;

[0055] Figure 2 The multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis method flow chart provided for the embodiments of the present application. DETAILED DESCRIPTION

[0056] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms without being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. It should be noted that the embodiments and features of the present application can be combined with each other unless there is a conflict. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0057] Referring to Figure 1 As shown in the drawings, in some embodiments of the present application, a multi-modal signal processing fetal movement and fetal abnormality feature diagnosis system comprises:

[0058] a signal acquisition module for acquiring multi-modal signals, the multi-modal signals comprising: a fetal heart signal, a fetal movement signal, and a uterine contraction signal;

[0059] a preprocessing module for removing noise and interference from the multi-modal signals;

[0060] a feature extraction module for extracting key features from the multi-modal signals, the key features comprising: frequency and amplitude;

[0061] a correlation analysis module for analyzing the temporal correlation of the key features of different modal signals;

[0062] a weight allocation module for dynamically allocating weights to the key features of different modal signals based on the temporal correlation of the key features of different modal signals;

[0063] a fusion judgment module for fusing the key features after dynamic weight allocation, and judging whether the fetus is abnormal based on the fused key features;

[0064] adjusting the extraction of the key features of the fetal heart signal based on the change period of the fetal heart signal and the fetal heart signal period threshold;

[0065] adjusting the extraction of the key features of the fetal movement signal based on the occurrence frequency of the fetal movement signal and the fetal movement signal frequency threshold;

[0066] adjusting the extraction of the key features based on the duration of the uterine contraction signal and the uterine contraction signal duration threshold;

[0067] determining the correlation between the key features of different modal signals based on the minimum time interval of changes between the key features of different modal signals.

[0068] Specifically, in the preprocessing module, when the noise intensity of the signal exceeds the preset noise intensity threshold, a filtering means is used for noise reduction operation; when the noise intensity of the signal does not exceed the preset noise intensity threshold, only simple signal smoothing processing is performed. The preset noise intensity threshold is an intensity value that can be used to distinguish obvious noise interference, which is determined by statistical analysis of the noise intensity of a large number of normal and abnormal fetal heart, fetal movement and uterine contraction signals.

[0069] It can be understood that, by collecting fetal heart, fetal movement and uterine contraction multi-modal signals, extracting key features after preprocessing, analyzing the time correlation of different modal characteristics and dynamically allocating weights, and then fusing to judge fetal abnormalities, the extraction strategy can be dynamically adjusted according to the characteristics of each signal, the implicit relationship between modalities can be deeply mined, the fusion accuracy can be improved through dynamic weight allocation, so that the fetal abnormality can be more accurately judged, and the defects of the prior art multi-modal fusion shallowization are effectively overcome.

[0070] In some embodiments of the present application, based on the change period of the fetal heart signal and the fetal heart signal period threshold, the extraction of the key features of the fetal heart signal is adjusted, including:

[0071] When the fetal heart signal change period is greater than the fetal heart signal period threshold, the frequency data less than the first frequency in the fetal heart signal is extracted;

[0072] When the fetal heart signal change period is less than or equal to the fetal heart signal period threshold, the frequency data greater than the first frequency in the fetal heart signal and the maximum amplitude and minimum amplitude in each first time interval are extracted;

[0073] Based on the occurrence frequency of the fetal movement signal and the fetal movement signal frequency threshold, the extraction of the key features of the fetal movement signal is adjusted, including:

[0074] When the fetal movement signal occurrence frequency is greater than the fetal movement signal frequency threshold, the occurrence frequency of the fetal movement signal in each second time interval and the amplitude value at each occurrence are extracted;

[0075] When the fetal movement signal occurrence frequency is less than or equal to the fetal movement signal frequency threshold, the amplitude value at each occurrence of the fetal movement signal and the time interval between two occurrences of the fetal movement signal are extracted;

[0076] Based on the duration of the uterine contraction signal and the uterine contraction signal duration threshold, the extraction of the key features is adjusted, including:

[0077] When the uterine contraction signal duration is greater than the uterine contraction signal duration threshold, the amplitude peak value of the uterine contraction signal and the time point at which the amplitude peak value appears are extracted;

[0078] When the duration of the uterine contraction signal is less than or equal to the duration threshold of the uterine contraction signal, the number of occurrences of the uterine contraction signal in every third time interval and the amplitude value at each occurrence are extracted.

[0079] Specifically, the state of the fetus in the uterus is not completely stable, and factors such as activity, sleep cycle conversion, and response to external stimuli can cause periodic changes in the frequency and amplitude of the fetal heart signal. For example, the heart rate of the fetus may increase when the fetus is active, and the heart rate may slow down when the fetus is sleeping. This periodic fast-slow change forms a change period of the fetal heart signal, which is not always stable. Considering the size relationship between the change period of the fetal heart signal and the fetal heart signal period threshold, different frequency features and amplitude features in different frequency bands are extracted from the fetal heart signal according to the fast-slow situation of the fetal heart signal change, so as to more accurately obtain key features reflecting the state of the fetus. The change period of the fetal heart signal has a corresponding relationship with the frequency feature. The fetal heart signal with a long change period usually contains more low-frequency band information, and the fetal heart signal with a short change period usually contains more high-frequency band information and more obvious amplitude changes.

[0080] When the change period of the fetal heart signal is less than or equal to the preset fetal heart signal period threshold, the signal changes relatively quickly. The maximum and minimum amplitude values in every first time interval are extracted, which can more accurately capture the rapid fluctuations of the fetal heart signal amplitude in this state, thereby comprehensively reflecting the amplitude features of the fetal heart signal in this state and providing more accurate amplitude information for subsequent analysis of the temporal relationship between key features of different modal signals.

[0081] When the fetal movement signal has a high occurrence frequency, the occurrence frequency and corresponding amplitude value are counted at a fixed second time interval, which can more efficiently reflect the regularity of its dense occurrence. When the occurrence frequency is low, the time interval between adjacent occurrences can more accurately reflect the rhythm of its sparse occurrence, and the key features of the fetal movement signal under different frequencies can be comprehensively captured by combining the amplitude features, thereby adapting to the characteristic differences of the fetal movement signal under different occurrence frequencies.

[0082] When the duration of the uterine contraction signal is greater than the preset duration threshold of the uterine contraction signal, the signal presents a more obvious single duration feature, and the extraction of the amplitude peak value and the time point at which the peak value occurs can effectively reflect its key characteristics. When the duration is short, the signal tends to occur multiple times in a short time. Therefore, by extracting the number of occurrences and amplitude features in every third time interval, the occurrence regularity and intensity information can be more comprehensively captured. The two extraction methods are respectively adapted to the feature performance of the uterine contraction signal under different durations.

[0083] The first frequency is a frequency demarcation value for distinguishing high and low frequencies of the fetal heart signal, which is determined by statistical analysis of the frequency range of normal fetal heart signals; the first time interval is a time interval value for extracting the amplitude feature of the fetal heart, which is determined by analysis of the amplitude variation law of normal fetal heart signals; the second time interval is a time interval value for counting the number of fetal movements, which is determined by analysis of the occurrence law of normal fetal movement signals; the fetal movement signal frequency threshold is a frequency value for distinguishing the density of fetal movement signal occurrence, which is determined by statistical analysis of the occurrence frequency of normal fetal movement signals; the uterine contraction signal duration threshold is a duration value for distinguishing the length of uterine contraction signal duration, which is determined by statistical analysis of the duration of normal uterine contraction signals; and the third time interval is a time interval value for counting the number of uterine contractions, which is determined by analysis of the occurrence law of normal uterine contraction signals.

[0084] It can be understood that, for fetal heart, fetal movement and uterine contraction signals, the extraction strategy of key features is dynamically adjusted according to the comparison between their own characteristics (variation period, occurrence frequency, duration) and the corresponding threshold value. This solves the problem of “one-size-fits-all” feature extraction of different modal signals in the prior art, makes the extracted key features more consistent with the actual state of the signal, provides more accurate and targeted basic data for the subsequent correlation analysis module to mine the implicit correlation between modalities, and avoids the superficialization of correlation analysis caused by mismatched feature extraction.

[0085] In some embodiments of the present application, when determining the correlation relationship between the key features of different modal signals based on the minimum time interval at which the key features of different modal signals change, the method comprises:

[0086] When the minimum time interval between the frequency variation time point of the fetal heart signal and the amplitude variation time point of the fetal movement signal is less than the first time interval, the correlation relationship between the frequency of the fetal heart signal and the amplitude of the fetal movement signal is a first correlation relationship;

[0087] When the minimum time interval between the frequency variation time point of the fetal heart signal and the amplitude variation time point of the fetal movement signal is greater than or equal to the first time interval, the correlation relationship between the frequency of the fetal heart signal and the amplitude of the fetal movement signal is a second correlation relationship;

[0088] When the minimum time interval between the frequency variation time point of the fetal heart signal and the amplitude peak occurrence time point or the amplitude variation time point of the uterine contraction signal is less than the second time interval, the correlation relationship between the frequency of the fetal heart signal and the amplitude or amplitude peak of the uterine contraction signal is a first correlation relationship;

[0089] when the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is greater than or equal to the second time interval, the correlation between the frequency of the fetal heart signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a second correlation;

[0090] when the minimum time interval between the amplitude change time point of the fetal movement signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is less than the third time interval, the correlation between the amplitude of the fetal movement signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a first correlation;

[0091] when the minimum time interval between the amplitude change time point of the fetal movement signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is greater than or equal to the third time interval, the correlation between the amplitude of the fetal movement signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a second correlation.

[0092] Specifically, the first time interval threshold is a threshold for judging the time closeness of the two by statistical determination of the time difference between the fetal heart rate feature change time point and the fetal movement amplitude feature change time point under a large number of normal physiological states of the fetus; the first correlation refers to the close connection in time between different modal characteristics, and the correlation state with high synchronization between the changes of the two; the second correlation refers to the relatively loose connection in time between different modal characteristics, and the correlation state with low synchronization between the changes of the two; the second time interval threshold is a threshold for judging the time closeness of the two by statistical determination of the time difference between the fetal heart rate feature change time point and the uterine contraction amplitude peak occurrence time point, uterine contraction amplitude change time point under a large number of normal physiological states of the fetus.

[0093] It can be understood that by defining the "minimum time interval" of the change of the key features of different modal signals, the first correlation (the time interval is less than a certain value) and the second correlation (the time interval is greater than or equal to a certain value) are clearly distinguished, for example, the first correlation when the time interval between the fetal heart rate change and the fetal movement amplitude change is less than the first time interval. This converts the ambiguous "inter-modal correlation" in the prior art into a specific relationship type that can be quantified and distinguished, realizes the fine capture of the time correlation between the modes, and more deeply excavates the time-dependent relationship between the modes than the simple judgment of "whether there is a correlation" in the prior art, providing specific and clear correlation basis for subsequent weight allocation.

[0094] In some embodiments of the present application, when the weights of the key features of different modal signals are dynamically allocated based on the time connection of the key features of different modal signals, the method comprises:

[0095] multiplying the number of the first association relationship between the key feature of the single modality signal and the key feature of the other modality signal by a weight coefficient corresponding to the first association relationship to obtain a first association relationship weighted value;

[0096] multiplying the number of the second association relationship between the key feature of the single modality signal and the key feature of the other modality signal by a weight coefficient corresponding to the second association relationship to obtain a second association relationship weighted value;

[0097] adding the first association relationship weighted value and the second association relationship weighted value to obtain an association relationship weighted sum;

[0098] multiplying the basic weight value of the key feature of the single modality signal by the association relationship weighted sum to obtain a basic weight association weighted product;

[0099] dividing the basic weight association weighted product by the sum of the number of the first association relationship and the number of the second association relationship to obtain the weight value of the key feature of the single modality signal.

[0100] Specifically, the weight coefficient corresponding to the first association relationship is used to measure the influence of the first association relationship on the key feature weight. Since the first association relationship reflects strong feature association, the weight coefficient corresponding to the first association relationship has a value greater than the weight coefficient corresponding to the second association relationship, which is a fixed constant. The weight coefficient corresponding to the second association relationship is used to measure the influence of the second association relationship on the key feature weight. Since the second association relationship reflects weak feature association, the weight coefficient corresponding to the second association relationship has a value less than the weight coefficient corresponding to the first association relationship, which is a fixed constant. The basic weight value of the key feature of the single modality signal is the initial weight reference of the single modality key feature, which is the starting point of subsequent dynamic adjustment of the weight, and is a preset value.

[0101] It can be understood that based on the number and corresponding coefficients of the first and second association relationships, the weight of the single modality key feature is calculated through a formula (such as the product of the association relationship weighted sum and the basic weight divided by the total number of associations). This changes the mode of fixed weight or subjective weight setting in the prior art, so that the weight distribution is directly linked to the number and type of actual associations between modalities - the closer the association (the more the first association relationship), the more the weight calculation result can reflect the importance of the feature. This dynamic weight distribution mechanism fully reflects the actual contribution of each modality feature in fusion, avoids the subjectivity of shallow weighted fusion, and deepens the depth of multi-modal fusion.

[0102] In some embodiments of the present application, the key features after the dynamic allocation of the weight are fused, and when it is judged whether the fetus has an abnormality based on the fused key features, the method comprises:

[0103] The key feature value of a single modality signal is multiplied by the weight value of the key feature of the corresponding single modality signal, to obtain a weighted key feature value;

[0104] All weighted key feature values are summed to obtain a fusion feature value;

[0105] When the fusion feature value is greater than the fusion feature threshold, it is determined that the fetus has an abnormality;

[0106] When the fusion feature value is less than or equal to the fusion feature threshold, it is determined that the fetus does not have an abnormality.

[0107] Specifically, the fusion feature threshold is determined based on multi-modality signal key feature fusion data of normal fetuses and multi-modality signal key feature fusion data of abnormal fetuses, and is dynamically adjusted according to the gestational age of the fetus.

[0108] It can be understood that the key features with dynamically assigned weights are fused (weighted summation to obtain a fusion feature value), and the fetus is determined to be abnormal based on the comparison of the fusion feature value and the threshold. This is different from the simple splicing feature or fixed weighting determination method in the prior art. The weight of each key feature in the fusion process is calculated based on the deep correlation between modalities, and the fusion feature value can comprehensively reflect the implicit correlation information between modalities. This fusion result is closer to the real correlation of the physiological state of the fetus, effectively improves the accuracy of abnormality determination, and solves the problem of insufficient determination accuracy caused by ignoring the inherent relationship between modalities in the existing shallow fusion.

[0109] Referring to Figure 2 In some embodiments of the present application, a multi-modality signal processing fetal movement and fetal heart abnormality feature diagnosis method includes:

[0110] S100, collecting multi-modality signals, the multi-modality signals including: fetal heart signals, fetal movement signals, and uterine contraction signals;

[0111] S200, removing noise and interference from the multi-modality signals;

[0112] S300, extracting key features from the multi-modality signals, the key features including: frequency and amplitude;

[0113] S400, analyzing the temporal relationship of the key features of different modalities;

[0114] S500, dynamically assigning weights to the key features of different modalities based on the temporal relationship of the key features of different modalities;

[0115] S600, the key features after the dynamic allocation weight are fused, and whether the fetus is abnormal is judged based on the fused key features.

[0116] It can be understood that the above-mentioned multi-modal signal processing fetal movement and fetal heart abnormality feature diagnosis method and system have the same beneficial effects, which will not be repeated here.

[0117] It should be noted that:

[0118] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0119] In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments but not others, the combination of features of different embodiments means within the scope of the present application and forms different embodiments.

[0120] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system, characterized in that, The method comprises the following steps: a signal acquisition module for acquiring multi-modal signals, the multi-modal signals including fetal heart signals, fetal movement signals, and uterine contraction signals; a preprocessing module for removing noise and interference from the multi-modal signals; a feature extraction module for extracting key features from the multi-modal signals, the key features including frequency and amplitude; a correlation analysis module for analyzing the correlation of the key features of different modal signals over time; a weight allocation module for dynamically allocating weights to the key features of different modal signals based on the correlation of the key features of different modal signals over time; a fusion judgment module for fusing the key features after dynamic weight allocation and judging whether the fetus is abnormal based on the fused key features; adjusting the extraction of the key features of the fetal heart signals based on the change period of the fetal heart signals and the fetal heart signal period threshold; adjusting the extraction of the key features of the fetal movement signals based on the occurrence frequency of the fetal movement signals and the fetal movement signal frequency threshold; adjusting the extraction of the key features based on the duration of the uterine contraction signals and the uterine contraction signal duration threshold; determining the correlation between the key features of different modal signals based on the minimum time interval at which the key features of different modal signals change.

2. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system as claimed in claim 1, wherein, When adjusting the extraction of the key features of the fetal heart signals based on the change period of the fetal heart signals and the fetal heart signal period threshold, the method comprises the following steps: when the fetal heart signal change period is greater than the fetal heart signal period threshold, extract the frequency data less than the first frequency in the fetal heart signal; when the fetal heart signal change period is less than or equal to the fetal heart signal period threshold, extract the frequency data greater than the first frequency in the fetal heart signal, and the maximum amplitude and minimum amplitude in each first time interval.

3. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system as claimed in claim 2, wherein, When adjusting the extraction of the key features of the fetal movement signals based on the occurrence frequency of the fetal movement signals and the fetal movement signal frequency threshold, the method comprises the following steps: when the fetal movement signal occurrence frequency is greater than the fetal movement signal frequency threshold, extract the occurrence frequency of the fetal movement signal in each second time interval and the amplitude value at each occurrence; when the fetal movement signal occurrence frequency is less than or equal to the fetal movement signal frequency threshold, extract the amplitude value at each occurrence of the fetal movement signal and the time interval between two occurrences of the fetal movement signal.

4. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system as claimed in claim 3, wherein, When adjusting the extraction of the key features based on the duration of the uterine contraction signals and the uterine contraction signal duration threshold, the method comprises the following steps: when the uterine contraction signal duration is greater than the uterine contraction signal duration threshold, extract the amplitude peak value of the uterine contraction signal and the time point at which the amplitude peak value appears; when the uterine contraction signal duration is less than or equal to the uterine contraction signal duration threshold, extract the occurrence frequency of the uterine contraction signal in each third time interval and the amplitude value at each occurrence.

5. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system as claimed in claim 4, wherein, When determining the correlation between the key features of different modal signals based on the minimum time interval at which the key features of different modal signals change, the method comprises the following steps: when the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude change time point of the fetal movement signal is less than a first time interval, then the correlation between the frequency of the fetal heart signal and the amplitude of the fetal movement signal is a first correlation; when the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude change time point of the fetal movement signal is greater than or equal to the first time interval, then the correlation between the frequency of the fetal heart signal and the amplitude of the fetal movement signal is a second correlation.

6. The fetal movement and fetal heart abnormality feature diagnosis system of multi-modal signal processing according to claim 5, characterized in that, when the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is less than a second time interval, then the correlation between the frequency of the fetal heart signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a first correlation; when the minimum time interval between the frequency change time point of the fetal heart signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is greater than or equal to the second time interval, then the correlation between the frequency of the fetal heart signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a second correlation.

7. The fetal movement and fetal heart abnormality feature diagnosis system of multi-modal signal processing according to claim 6, characterized in that, when the minimum time interval between the amplitude change time point of the fetal movement signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is less than a third time interval, then the correlation between the amplitude of the fetal movement signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a first correlation; when the minimum time interval between the amplitude change time point of the fetal movement signal and the amplitude peak occurrence time point of the uterine contraction signal or the amplitude change time point of the uterine contraction signal is greater than or equal to the third time interval, then the correlation between the amplitude of the fetal movement signal and the amplitude of the uterine contraction signal or the amplitude peak of the uterine contraction signal is a second correlation.

8. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system as claimed in claim 7, wherein, When the correlation between the key features of different modal signals is dynamically assigned weights in time, it includes: the first correlation weighting value is obtained by multiplying the number of first correlations between the key features of a single modal signal and the key features of other modal signals by the weight coefficient corresponding to the first correlation; the second correlation weighting value is obtained by multiplying the number of second correlations between the key features of a single modal signal and the key features of other modal signals by the weight coefficient corresponding to the second correlation; the correlation weighting total sum is obtained by adding the first correlation weighting value and the second correlation weighting value; the basic weight correlation weighted product is obtained by multiplying the basic weight value of the key features of a single modal signal by the correlation weighting total sum; The basic weight correlation weighted product is divided by the sum of the first correlation quantity and the second correlation quantity to obtain a weight value of the key feature of the single modality signal.

9. A multi-modal signal processing fetal movement fetal heart abnormality feature diagnostic system as claimed in claim 8, wherein, The dynamically allocated weighted key features are fused, and whether the fetus is abnormal is determined based on the fused key features. The key feature values of the single modality signals are multiplied by the weight values of the key features of the respective single modality signals to obtain weighted key feature values. All the weighted key feature values are summed to obtain a fusion feature value. When the fusion feature value is greater than a fusion feature threshold value, it is determined that the fetus is abnormal. When the fusion feature value is less than or equal to the fusion feature threshold value, it is determined that the fetus is normal.

10. A method of fetal movement-fetal heart abnormality feature diagnosis of multi-modal signal processing, characterized in that, The application is applied to a multi-modality signal processing fetal movement and fetal heart abnormality feature diagnosis system according to any one of claims 1-9, comprising: S100, collecting multi-modality signals, wherein the multi-modality signals comprise fetal heart signals, fetal movement signals, and uterine contraction signals; S200, removing noise and interference from the multi-modality signals; S300, extracting key features from the multi-modality signals, wherein the key features comprise frequencies and amplitudes; S400, analyzing the time correlation of the key features of different modality signals; S500, dynamically allocating weights to the key features of different modality signals based on the time correlation of the key features of different modality signals; S600, fusing the dynamically allocated weighted key features, and determining whether the fetus is abnormal based on the fused key features.

Images