Multi-mode-based urodynamics evaluation method and related equipment

Abstract

The invention discloses a urodynamics evaluation method based on multiple modes and related equipment. The method comprises the steps that multi-source urodynamics information is obtained, and the multi-source urodynamics information comprises urodynamics signals, urodynamics clinical characteristics and patient personal information; based on the personal information of the patient, correcting the urodynamic signals and the urodynamic clinical features through a label offset atlas model; and inputting the personal information of the patient, the corrected urodynamics signal and the corrected urodynamics clinical characteristics into a urodynamics evaluation model to obtain a urodynamics evaluation result of the patient. The problems that the urodynamic assessment process mainly depends on collaborative observation and analysis of urodynamic examination by clinicians, the degree of dependence on professional knowledge and clinical experience of the clinicians is large, no clear diagnostic standard exists in the urodynamic examination, and the diagnostic result is subjective can be solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of intelligent medical treatment, and more specifically, to a multi-modal based urodynamic evaluation method and related equipment. BACKGROUND

[0002] Currently, in clinical practice, urodynamic evaluation mainly relies on the real-time observation and comprehensive judgment of multiple physiological parameters by physicians during UDS (Urodynamics) process, including the change trend of key indicators such as bladder pressure, abdominal pressure, detrusor pressure, urine flow rate, bladder volume, etc. Doctors usually manually interpret the shape, timing characteristics and interaction of these pressure and flow rate curves to determine the lower urinary tract function status of the patient. However, this process is highly dependent on the professional knowledge and experience of the clinician, and different physicians may give different interpretations of the same set of curve data, which is highly subjective and has poor consistency. Moreover, the diagnostic criteria for urodynamics have not yet been standardized, resulting in a lack of objectivity and repeatability in overall evaluation, which seriously affects the credibility of the test results and the standardization of clinical application. SUMMARY

[0003] A series of simplified concepts are introduced in the summary section, which will be further described in detail in the specific embodiments section. The summary section of the present application does not mean to attempt to limit the key features and essential technical features of the claimed technical solutions, nor does it attempt to determine the protection scope of the claimed technical solutions.

[0004] To solve the problem that the urodynamic evaluation process mainly relies on the observation and analysis of urodynamic examination by clinicians, and the dependence on the professional knowledge and clinical experience of the doctor is high, and the urodynamic examination has no clear diagnostic criteria, and the diagnostic results are subjective, in a first aspect, the present application proposes a multi-modal based urodynamic evaluation method, the method comprising:

[0005] Obtaining multi-source urodynamic information, wherein the multi-source urodynamic information includes urodynamic signals, urodynamic clinical features, and patient personal information;

[0006] Correcting the urodynamic signals and urodynamic clinical features based on the patient personal information through a label offset atlas model;

[0007] Inputting the patient personal information and the corrected urodynamic signals and urodynamic clinical features into a urodynamic evaluation model to obtain the urodynamic evaluation result of the patient.

[0008] Optionally, it further comprises:

[0009] Identify the theoretical time point corresponding to the physiological stage of different patients and the subjective time point actually fed back by the patient based on the actually measured urodynamic signals of different patients;

[0010] Determine the label offset based on the theoretical time point and the subjective time point, and construct an offset time training sample set based on the patient personal information of different patients.

[0011] Optionally, it further comprises:

[0012] Use a linear regression model to fit the relationship between the patient feature data and the label offset, train the model weight on the offset time training sample set by minimizing the loss function, and obtain a label offset atlas model; or,

[0013] Use a multilayer perception model to train and learn the nonlinear mapping of the patient feature data and the label offset on the offset time training sample set, to obtain a label offset atlas model.

[0014] Optionally, it further comprises:

[0015] Reform the label offset atlas model to make the label offset atlas model output the mean and standard deviation of the label offset distribution;

[0016] Use a loss encouragement model:

[0017]

[0018] Train the reformed label offset atlas model to make the model output the credibility of the offset prediction value at the same time, where N is the total number of samples, j is the index variable, δ j is the actual offset value, μ j is the expected value of the predicted offset, σ j is the standard deviation of the predicted offset.

[0019] Optionally, the urodynamic evaluation model utilizes a convolutional neural network and an expert knowledge system to achieve.

[0020] Optionally, it further comprises:

[0021] Collect time domain features and / or frequency domain features for multi-source urodynamic information to obtain time domain features and / or frequency domain features;

[0022] Information encoding is performed on the time domain features and / or frequency domain features;

[0023] Feature fusion is performed on the information encoded features to obtain a fusion feature signal, the convolutional neural network comprises an input layer, a convolutional layer, a pooling layer, a fully connected layer, and a Softmax layer, and the expert knowledge system comprises an expert knowledge base and a reasoning machine.

[0024] characteristics of the urodynamic signal are extracted by using the input layer, the convolutional layer and the pooling layer to obtain at least one feature;

[0025] based on the at least one feature, the evaluation result is generated by using the full connection layer and the Softmax layer;

[0026] the clinical characteristics and the personal information are input into the expert knowledge system to output an expert evaluation result;

[0027] the intelligent evaluation result and the expert evaluation result are decision-fused to output a final diagnosis result.

[0028] Optionally, the method further comprises:

[0029] the urodynamic signal in the training sample set is embedded in a low dimension and reconstructed by using a variational autoencoder, and a reconstruction error is calculated;

[0030] a pseudo-normal sample is identified based on a preset abnormal threshold, the pseudo-normal sample being a sample that is labeled as normal but has an abnormal reconstruction error;

[0031] the pseudo-normal sample is removed from the original training sample set;

[0032] a network structure having three sub-model branches is constructed to learn normal samples, abnormal samples and pseudo-normal samples, respectively;

[0033] a gating network structure is constructed to receive feature information of any input sample and output weight values corresponding to the three sub-model branches, the weight values being used to indicate which sub-model branch is more suitable for processing the current sample;

[0034] the branch weights output by the gating network are weighted and fused with prediction results of the three sub-models to obtain a final prediction result;

[0035] based on errors between class labels of samples and the fused prediction results, parameters of the gating network and the three sub-model branches are jointly optimized to complete a urodynamic evaluation model training process.

[0036] In a second aspect, the present application further provides a urodynamic evaluation device based on multiple modalities, comprising:

[0037] an acquisition unit configured to acquire multi-source urodynamic information, wherein the multi-source urodynamic information comprises urodynamic signals, urodynamic clinical characteristics and patient personal information;

[0038] a correction unit configured to correct the urodynamic signals and the urodynamic clinical characteristics by using a label offset atlas model based on the patient personal information;

[0039] an evaluation unit configured to input the patient personal information and the corrected urodynamic signal and urodynamic clinical feature into a urodynamic evaluation model to obtain a urodynamic evaluation result of the patient.

[0040] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor is configured to implement the steps of the multi-modal based urodynamic evaluation method according to any one of the first aspect when executing the computer program stored in the memory.

[0041] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the multi-modal based urodynamic evaluation method according to any one of the first aspect.

[0042] In summary, the multi-modal based urodynamic evaluation method provided by the present application, by acquiring multi-source urodynamic information, wherein the multi-source urodynamic information includes urodynamic signal, urodynamic clinical feature, and patient personal information, correcting the urodynamic signal and urodynamic clinical feature based on the patient personal information through a label shift atlas model, and inputting the patient personal information and the corrected urodynamic signal and urodynamic clinical feature into a urodynamic evaluation model to obtain a urodynamic evaluation result of the patient. Thus, first, the introduction of the label shift atlas effectively overcomes the interference of the patient subjective labeling time point error on the model training and reasoning accuracy in the urodynamic examination, improves the individual precision of event recognition and label alignment; second, by modeling the multi-modal information, not only the original signal shape information is introduced, but also the clinical quantitative features and individual background factors are considered, so that the model has a more global perspective and the diagnosis is more comprehensive and robust; finally, the method has good scalability and interpretability, and can output the corrected time of key events, key features affecting diagnosis, and individual risk level in the reasoning stage, providing visual and quantifiable auxiliary judgment basis for clinicians, reducing the dependence on artificial experience, and improving the intelligent level of urodynamic examination auxiliary evaluation.

[0043] The multi-modal based urodynamic evaluation method of the present application, other advantages, objects and features of the present application will be embodied in part by the following description, and will be understood by those skilled in the art through research and practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0044] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments with reference made to the accompanying drawings. The drawings are for purposes of illustration only and are not intended to be limiting in

[0045] Figure 1 A multi-modal based urodynamic evaluation method provided by an embodiment of the present application has a flowchart as shown in FIG. 1.

[0046] Figure 2 A multi-modal based urodynamic evaluation device provided by an embodiment of the present application has a structure diagram as shown in FIG. 2.

[0047] Figure 3 A multi-modal based urodynamic evaluation electronic device provided by an embodiment of the present application has a structure diagram as shown in FIG. 3. DETAILED DESCRIPTION

[0048] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, if any, are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the use of these terms herein is to be construed as interchangeable in order to distinguish between the similar objects. It is also to be understood that the terms "comprising", "having", "including" and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to") unless otherwise noted. Embodiments of the present application will now be described, by way of example only, with reference to the attached schematic drawings, in which:

[0049] To solve the problem that the urodynamic evaluation process mainly relies on the observation and analysis of urodynamic examination by the clinician, and the dependence on the professional knowledge and clinical experience of the doctor is high, and there is no clear diagnostic standard for UDS, and the diagnostic result is subjective, please refer to Figure 1 A multi-modal based urodynamic evaluation method provided by an embodiment of the present application has a flowchart as shown in FIG. 1. Specifically, the method can include steps S110 to S130.

[0050] S110, acquiring multi-source urodynamic information, wherein the multi-source urodynamic information includes urodynamic signals, urodynamic clinical features, and patient personal information.

[0051] S120, correcting the urodynamic signal and the urodynamic clinical feature based on the patient personal information through a label offset graph model.

[0052] S130, inputting the patient personal information and the corrected urodynamic signal and urodynamic clinical feature into a urodynamic evaluation model to obtain a urodynamic evaluation result of the patient.

[0053] Exemplarily, first, multi-source urodynamic information of a patient is acquired, including three main components: first, urodynamic signal data, usually time series signals, including bladder pressure, abdominal pressure, detrusor pressure (calculated from the former two) and urine flow rate, etc.; second, urodynamic clinical feature data, including maximum bladder capacity, bladder compliance, detrusor peak pressure, residual urine volume, etc., which are numerical or rating data recorded by the examination system; and third, personal information of the patient, including age, gender, whether there is diabetes, previous operation history, pelvic floor function score, anxiety score, etc., which are clinical background information that will significantly affect the subjective perception time point and physiological response of the patient to the urination process.

[0054] Exemplarily, after the above multi-source information is acquired, the system first corrects the key event points (such as “initial urge to urinate”, “intense urge to urinate”, etc.) in the original urodynamic signal and clinical feature based on the patient personal information through a constructed label offset graph model. The offset graph model is a machine learning model constructed based on a large amount of historical patient data, which is used to learn the relationship between the individual characteristics of the patient and the time error of the subjective labeled event. For the current patient, the model can output the predicted offset value of the key event, and correct the original event time point accordingly to be closer to the physiological real trigger time. For example, for patients who are older or have nervous system diseases, their perception of detrusor contraction may be significantly delayed, and the system can predict that there is a 5-10 second offset of the subjective labeling time point relative to the actual physiological event, and perform a backward shift processing on the key feature extraction window in the signal processing process, thereby significantly improving the fitting accuracy of the subsequent evaluation model on the corresponding relationship between the signal features and the real pathological state.

[0055] Exemplarily, after the offset correction is completed, the system encodes and fuses the personal information of the patient, the corrected urodynamic signal data and the clinical feature data into a unified feature vector as the input of the evaluation model. Specifically, for time series type signal data such as bladder pressure and urine flow rate, a convolutional network or a time sequence Transformer structure can be used to extract its key time domain pattern; for structured clinical features and personal information, normalization, embedding layer, fully connected layer and other methods can be used for encoding, and finally a unified input vector is formed through feature splicing. The evaluation model can use a multilayer perception or a lightweight graph neural network model structure to complete the identification of urodynamic abnormalities and the prediction of risk scores.

[0056] In summary, the urodynamic evaluation method based on multiple modalities provided by the embodiments of the present application acquires multiple source urodynamic information, wherein the multiple source urodynamic information includes urodynamic signal, urodynamic clinical feature and patient personal information; corrects the urodynamic signal and the urodynamic clinical feature based on the patient personal information through a label offset atlas model; inputs the patient personal information and the corrected urodynamic signal and urodynamic clinical feature into a urodynamic evaluation model to obtain the urodynamic evaluation result of the patient. Thus, first, the introduction of the label offset atlas effectively overcomes the interference of the patient subjective labeling time point error on the model training and reasoning accuracy in the urodynamic examination, improves the individual accuracy of event recognition and label alignment; second, by fusing and modeling multiple modalities, not only the original signal form information is introduced, but also the clinical quantitative features and individual background factors are considered, so that the model has a more global perspective and the diagnosis is more comprehensive and robust; finally, the method has good scalability and interpretability, and can output the corrected time of key events, key features affecting diagnosis and individual risk level in the reasoning stage, providing visual and quantifiable auxiliary judgment basis for clinicians, reducing the dependence on artificial experience, and improving the intelligent level of urodynamic examination auxiliary evaluation.

[0057] In some examples, further comprising:

[0058] Identifying the theoretical time point corresponding to the physiological state of the patient at different urine intention stages and the subjective time point actually fed back by the patient based on the urodynamic signal actually measured by different patients;

[0059] Constructing an offset time training sample set based on the label offset determined according to the theoretical time point and the subjective time point and the patient feature data extracted based on the patient personal information of different patients.

[0060] In some examples, the system further comprises the following steps before constructing the label shift atlas model for achieving personalized correction of urodynamic signals: identifying key physiological event points in the process of micturition based on the actual measured urodynamic signals of different patients, such as the first detrusor excitation point, the rapid rise point of bladder pressure, the initial point of urinary flow rate, etc., which can represent the theoretical trigger time points of different micturition stages of the patient; at the same time, collecting the subjective feedback time points such as "initial micturition" and "intense micturition" recorded by the patient during the examination process through buttons or other means. The system calculates the label shift amount of each patient by comparing the above theoretical time points with the subjective annotation time points, i.e. the delay or advance time of subjective annotation relative to the physiological real event.

[0061] The system models the above label shift amount and the individual characteristic information of the patient. Structured features related to micturition perception delay are extracted from the patient's personal information, including but not limited to: age, gender, pelvic floor function score, anxiety state score, history of diabetes, history of spinal surgery, central or peripheral nerve damage history, history of urological surgery, etc., and this set of individual characteristics and the corresponding label shift amount form a training sample to construct the training set of the shift time prediction model. The training set can be used to supervise the training of the machine learning model to establish the mapping relationship between individual characteristics and shift time. The shift prediction model can output the event shift estimation value of the current patient, and can be further used to align and correct its urodynamic signals and clinical characteristics. For example, when evaluating a patient who is older and accompanied by pelvic floor muscle weakness (such as postmenopausal women), the model may predict that the subjective annotation event is delayed by 6 seconds relative to the actual event, then the system will automatically shift the signal analysis window backward by 6 seconds based on the estimation result to extract a signal segment that is more consistent with physiological changes, thereby improving the accuracy and consistency of the subsequent urodynamic evaluation model in event recognition and classification. The shift modeling and correction mechanism effectively compensates for the label misplacement problem caused by the difference between physiology and perception among patients, significantly enhances the individual adaptability of the model, reduces the structural error in the training data, and improves the recognition sensitivity of boundary samples and the reliability of clinical application of the evaluation system based on urodynamic signals.

[0062] In some examples, the system further comprises the following steps before constructing the label shift atlas model for achieving personalized correction of urodynamic signals: identifying key physiological event points in the process of micturition based on the actual measured urodynamic signals of different patients, such as the first detrusor excitation point, the rapid rise point of bladder pressure, the initial point of urinary flow rate, etc., which can represent the theoretical trigger time points of different micturition stages of the patient; at the same time, collecting the subjective feedback time points such as "initial micturition" and "intense micturition" recorded by the patient during the examination process through buttons or other means. The system calculates the label shift amount of each patient by comparing the above theoretical time points with the subjective annotation time points, i.e. the delay or advance time of subjective annotation relative to the physiological real event.

[0063] fitting the relationship between the patient characteristic data and the label shift amount using a linear regression model, training the model weights on the shift time training sample set by minimizing a loss function to obtain the label shift atlas model; or,

[0064] training the patient characteristic data and the label shift amount on the shift time training sample set using a multilayer perception model to obtain the label shift atlas model.

[0065] For example, to establish an individualized correction mechanism for the labeling of urinary desire events, the system first determines the physiological reference time points of different urinary desire stages by identifying key physiological event points in the actually measured urodynamic signals of the patient (such as the onset of detrusor excitation, the turning point of bladder pressure rise, or the first rise point of urinary flow rate). At the same time, the subjective labeling time points provided by the patient during the examination through the button or voice feedback, such as "initial urinary desire" or "strong urinary desire", are recorded. The system compares these two types of time points and calculates the label offset of each patient, that is, the time difference between the subjective labeling event and the real physiological triggering event.

[0066] For example, the system extracts patient individual feature information related to the offset, including but not limited to age, gender, pelvic floor muscle strength score, history of diabetes, history of nervous system or urinary system surgery, anxiety score, etc. These structured features and corresponding offsets together constitute an offset time training sample set, which is used as training data for the label offset atlas model. The label offset atlas model can be constructed based on a linear regression method. Specifically, the patient feature vector is taken as the input variable and the label offset is taken as the target output, and the linear relationship between them is fitted. The linear model minimizes the mean square error loss function:

[0067]

[0068] where N is the total number of samples, i is the sample index, δ j is the actual offset value, x i is the patient feature vector, and the weight parameter w is optimized on the offset time training sample set to obtain a regression model that can predict the offset for any new patient input feature.

[0069] In other examples, to model more complex nonlinear offset behavior, a multilayer perceptron model can be used to construct the label offset atlas model. The multilayer perceptron model usually contains one or more layers of nonlinear activation hidden layers. Through forward propagation and back propagation training on the offset training sample set, the high-dimensional nonlinear mapping relationship between the patient features and the label offset is learned. This method can be applied to complex scenarios where the label offset is affected by multiple factors, such as female patients with neurogenic bladder after the point of no return. The subjective urinary desire response time often shows a nonlinear offset trend due to the synergistic effect of multiple variables.

[0070] Exemplarily, after the training is completed, whether it is a linear model or a multi-layer perception model, the individual feature input of a new patient can be inferred to predict the label offset time, so as to correct the time alignment of the urodynamic signal and the event label of the new patient, and thus provide more accurate input data for the evaluation model. The correction mechanism not only improves the accuracy of the model in identifying events, but also improves the general adaptability of the model to different types of patients, and has good individual difference tolerance and clinical applicability.

[0071] In some examples, further comprising:

[0072] The label offset atlas model is modified to make the label offset atlas model output the mean and standard deviation of the label offset distribution;

[0073] The loss encourages the model to:

[0074]

[0075] The modified label offset atlas model is trained to make the model output the confidence of the offset prediction value simultaneously, where N is the total number of samples, j is the index variable, δ j is the actual offset value, μ j is the expected value of the predicted offset, and σ j is the standard deviation of the predicted offset.

[0076] It can be understood that the first term is used to measure the distance between the predicted value μ j and the true offset δ j , and the larger the standard deviation, the wider the fault tolerance; the second term logσ j is a regularization term to prevent the model from reducing the first term by infinitely amplifying σ j ; the modified offset prediction model can output the mean μ j and the standard deviation σ j of the offset simultaneously, forming a double-output structure. The system uses a training set containing patient individual features and actual offset labels to train the model, which can guide the model to give greater error tolerance when facing samples with high offset uncertainty, and to make more accurate fitting on samples with stable offset, thereby improving the robustness and interpretability of the overall offset prediction. The loss encourages the model to tolerate high uncertainty samples while accurately fitting low uncertainty samples, improving overall robustness.

[0077] In some examples, the urodynamic evaluation model utilizes a convolutional neural network and an expert knowledge system to achieve.

[0078] In some examples, further comprising:

[0079] The time domain features and / or frequency domain features of the multi-source urodynamic information are collected to obtain time domain features and / or frequency domain features;

[0080] The time domain features and / or frequency domain features are information coded;

[0081] The information coded features are fused to obtain a fused feature signal, the convolutional neural network comprises an input layer, a convolutional layer, a pooling layer, a fully connected layer, and a Softmax layer, and the expert knowledge system comprises an expert knowledge base and a reasoning machine;

[0082] The urodynamic signal is feature extracted by using the input layer, the convolutional layer, and the pooling layer to obtain at least one feature;

[0083] Based on the at least one feature, the evaluation result is generated by using the fully connected layer and the Softmax layer;

[0084] The clinical features and personal information are input into the expert knowledge system to output an expert evaluation result;

[0085] The intelligent evaluation result and the expert evaluation result are decision fused to output a final diagnosis result.

[0086] Exemplarily, the input layer is configured to input the feature fusion result, for example, the feature fusion result can be input in the form of an image. The convolutional layer is configured to extract image features from the feature fusion result input in the form of an image. The pooling layer is configured to select the image features extracted in the convolutional layer. Specifically, the pooling operation can be max pooling, average pooling, etc. On the one hand, the pooling layer can select image information that is not disturbed by position, and on the other hand, the pooling layer can reduce the dimension of the image features, thereby selecting important image features. Finally, based on the important image features selected by the pooling layer, the intelligent diagnosis classification is performed through the fully connected layer and the Softmax layer. Specifically, the fully connected layer is configured to convert all the important image features of the pooling layer into a plurality of diagnosis classification results, and then the Softmax layer is configured to convert the plurality of diagnosis results into probabilities, thereby generating the urodynamics evaluation result. The above method extracts at least one feature from the multi-source signals by using the input layer, the convolutional layer and the pooling layer, and generates the evaluation result by using the fully connected layer and the Softmax layer based on the at least one feature, thereby realizing intelligent, automatic and scientific training of the urodynamics evaluation model and eliminating the influence of human subjective factors on the urodynamics evaluation result. The expert knowledge system includes an expert knowledge base and a reasoning machine. The urodynamics clinical features and the personal information of the patient are input into the expert knowledge system. The input information is matched in the expert knowledge base, and the expert evaluation result is obtained by logical reasoning. Based on the evaluation result, the related parameters of the urodynamics evaluation model are adjusted, including: the professional physician analyzes the evaluation result to obtain an analysis result; and based on the analysis result, the parameters of each layer of the convolutional neural network are dynamically adjusted. The expert can label the analysis result to obtain an analysis result, and the analysis result can be used to further train the urodynamics evaluation model by dynamically adjusting the parameters of each layer of the convolutional neural network and the rules of the expert knowledge base. The above method continuously upgrades the urodynamics evaluation system by accumulating a large number of expert analysis results, thereby improving the accuracy of the urodynamics evaluation result.

[0087] Exemplarily, a digital bladder model of a virtual space can be established based on a real bladder, and the digital bladder is visually displayed by using a user interface, and then the urodynamics evaluation result is displayed. The above method realizes the visual display of the urodynamics evaluation result by configuring the user interface to display the urodynamics evaluation result.

[0088] According to some embodiments, further comprising:

[0089] The urodynamics signals in the training sample set are embedded in a low dimension and reconstructed by the variational autoencoder, and the reconstruction error is calculated;

[0090] The pseudo-normal samples are identified based on a preset abnormal threshold, and the pseudo-normal samples are samples that are labeled as normal but have abnormal reconstruction error;

[0091] remove pseudo-normal samples from the original training sample set;

[0092] a network structure with three sub-model branches is constructed to learn normal samples, abnormal samples and pseudo-normal samples respectively;

[0093] A gating network structure is constructed to receive the feature information of any input sample and output the weight values corresponding to the three sub-model branches, which are used to indicate which sub-model branch is more suitable for processing the current sample;

[0094] The branch weights output by the gating network are weighted and fused with the prediction results of the three sub-models to obtain the final prediction result;

[0095] Based on the error between the class label of the sample and the fusion prediction result, the parameters of the gating network and the three sub-model branches are optimized jointly to complete the training process of the urodynamic evaluation model.

[0096] It can be understood that, in order to further improve the expression ability and boundary recognition accuracy of the urodynamic evaluation model for complex sample structures, the present application introduces a pseudo-normal sample recognition mechanism and a multi-branch mixed expert structure optimization mechanism in the model training stage.

[0097] For example, first, the urodynamic signals (such as bladder pressure, detrusor pressure, urine flow rate, etc. time series data) in the training sample set are processed by introducing VAE (Variational AutoEncoder, Variational AutoEncoder) to map high-dimensional time series signals to low-dimensional latent space representation, and then reconstruct the original signal. By comparing the error (such as mean square reconstruction error or reconstruction loss) between the original signal and the reconstructed signal, the reconstruction error score of each sample is calculated. Then, based on statistical rules or set abnormal judgment strategy (such as 95% quantile value), set the abnormal threshold to identify pseudo-normal samples, that is, those samples labeled as normal in the training set but with significantly higher reconstruction error than the normal distribution range. Such samples may be close to normal in signal form, but in essence there are functional disorders or potential pathological changes. Directly including normal class modeling will have a negative impact on model generalization.

[0098] Exemplarily, the identified pseudo-normal samples can adopt two processing strategies, one of which is to be removed from the original training set and not participate in the standard positive and negative class model training; the other is the strategy adopted in the embodiment, that is, a sub-model branch is separately set up for the pseudo-normal samples, and a three-branch multi-expert network structure is constructed, including: a first sub-model for learning the feature mode of typical normal samples, a second sub-model for learning typical abnormal samples, and a third sub-model for learning the unique behavior characteristics and boundary mode in pseudo-normal samples. The structure can be based on a shared backbone network followed by three exclusive output heads or configure an independent network structure for each sub-model.

[0099] Exemplarily, in order to realize the dynamic routing allocation of the model to the input samples in the inference stage, a gating network structure can be introduced. The gating network receives the fusion feature vector input of the sample and outputs three groups of weight values, respectively corresponding to the three sub-model branches. Each weight represents the probability or adaptation degree of the current input sample being processed by which sub-model. In the inference stage, the three sub-models respectively evaluate the same input sample and output their respective prediction results. The system weights and fuses the three groups of prediction results according to the weight values output by the gating network to generate the final evaluation result.

[0100] Exemplarily, in the training stage, the actual label of the sample is compared with the above fusion prediction result, and the error thereof is used as a loss signal to jointly optimize all parameters of the gating network and the three sub-model branches, that is, the routing decision ability and the expression ability of each branch model are simultaneously adjusted through the back propagation mechanism. This training method ensures that the model not only can identify regular typical samples, but also has the ability to identify edge fuzzy samples (such as functionally abnormal but morphologically normal samples), thereby improving the robustness and individual adaptation ability of the overall model in complex urodynamic behavior scenarios.

[0101] It can be understood that by introducing pseudo-normal identification and multi-branch expert network structure, the model boundary ambiguity problem caused by the mixing of functionally abnormal but appearance normal samples into the normal category in the traditional evaluation model is effectively solved, and the classification accuracy of the model for boundary samples and mixed mode samples is improved. The introduction of the gating mechanism not only enhances the dynamic adaptation ability of the model in the inference stage, but also improves the expression range of the entire system for complex and diverse urodynamic behavior patterns, thereby providing more refined and individualized evaluation basis for complex patient types in actual clinical practice.

[0102] For details, please refer to Figure 2 An embodiment of the multi-modal based urodynamic evaluation device in the embodiment of the present application can include:

[0103] An acquisition unit is configured to acquire multi-source urodynamic information, wherein the multi-source urodynamic information includes urodynamic signals, urodynamic clinical characteristics, and patient personal information.

[0104] a correction unit configured to correct the urodynamic signal and the urodynamic clinical feature based on the patient personal information by a label shift graph model;

[0105] an evaluation unit configured to input the patient personal information and the corrected urodynamic signal and urodynamic clinical feature into a urodynamic evaluation model to obtain a urodynamic evaluation result of the patient.

[0106] In summary, the urodynamic evaluation device based on multiple modalities provided by the embodiments of the present application obtains multiple-source urodynamic information, wherein the multiple-source urodynamic information includes a urodynamic signal, a urodynamic clinical feature, and patient personal information; corrects the urodynamic signal and the urodynamic clinical feature based on the patient personal information by a label shift graph model; and inputs the patient personal information and the corrected urodynamic signal and urodynamic clinical feature into a urodynamic evaluation model to obtain a urodynamic evaluation result of the patient. Thus, first, the introduction of the label shift graph effectively overcomes the interference of the patient subjective labeling time point error on the model training and inference accuracy in the urodynamic examination, and improves the individual precision of event recognition and label alignment; second, by modeling the fusion of multiple modalities, not only the original signal form information is introduced, but also the clinical quantitative features and individual background factors are considered, so that the model has a more global perspective and the diagnosis is more comprehensive and robust; finally, the method has good scalability and interpretability, and can output the corrected time of the key event occurrence, the key features affecting the diagnosis, and the individual risk level in the inference stage, thereby providing visual and quantifiable auxiliary judgment basis for the clinician, reducing the dependence on artificial experience, and improving the intelligent level of urodynamic examination auxiliary evaluation.

[0107] As shown in Figure 3 The embodiments of the present application also provide an electronic device 300, which includes a memory 310, a processor 320, and a computer program 311 stored in the memory 320 and executable on the processor, and the processor 320 implements the steps of any method of the above-mentioned urodynamic evaluation based on multiple modalities when executing the computer program 311.

[0108] Since the electronic device introduced in the embodiments is a device used to implement the urodynamic evaluation device based on multiple modalities in the embodiments of the present application, the specific implementation of the electronic device and its various forms can be understood by those skilled in the art based on the methods introduced in the embodiments of the present application, so the implementation of the method in the embodiments of the present application by the electronic device will not be described in detail here, and any device used to implement the method in the embodiments of the present application belongs to the scope of protection of the present application.

[0109] In a specific implementation, the computer program 311, when executed by a processor, can implement Figure 1 Any of the embodiments described above.

[0110] It should be noted that in the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0111] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0112] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate one or more functions implemented in the flowcharts and / or block diagrams. Figure 1 One or more functions in a flow or a plurality of flows and / or blocks. Figure 1 An apparatus that performs the functions specified in one or more blocks or flows.

[0113] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions in the flowcharts and / or block diagrams. Figure 1 One or more functions in a flow or a plurality of flows and / or blocks. Figure 1 An apparatus that performs the functions specified in one or more blocks or flows.

[0114] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions in the flowcharts and / or block diagrams. Figure 1 One or more functions in a flow or a plurality of flows and / or blocks. Figure 1 An apparatus that performs the functions specified in one or more blocks or flows.

[0115] The embodiments of the present application further provide a computer program product, which comprises computer software instructions, and when the computer software instructions are run on a processing device, the processing device is caused to perform the flow or function according to the embodiments of the present application. Figure 1 Flow of the multi-modal based urodynamic evaluation in the corresponding embodiments.

[0116] The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function according to the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that the computer can store or the data storage device such as server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0118] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the above-described device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0120] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0121] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various program code storage media.

[0122] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A multimodal urodynamic assessment method, characterized in that, include: Acquire multi-source urodynamic information, wherein the multi-source urodynamic information includes urodynamic signals, urodynamic clinical characteristics, and patient personal information; Based on the patient's personal information, the urodynamic signals and clinical characteristics were corrected using a label offset mapping model; The patient's personal information, along with the corrected urodynamic signals and clinical urodynamic characteristics, are input into the urodynamic assessment model to obtain the patient's urodynamic assessment results.

2. The method as described in claim 1, characterized in that, Also includes: Based on the urodynamic signals actually measured by different patients, the theoretical time points corresponding to the physiological stages of different urination urges and the subjective time points actually reported by the patients are identified. An offset time training sample set is constructed based on the label offset determined by the theoretical time point and the subjective time point, as well as the patient feature data extracted based on the patient's personal information of different patients.

3. The method as described in claim 2, characterized in that, Also includes: A linear regression model is used to fit the relationship between patient feature data and label offsets. The model weights are trained on the training sample set at the offset time by minimizing the loss function to obtain the label offset map model; or... A multilayer perceptron model is used to train on the offset time training sample set to learn a nonlinear mapping between patient feature data and label offsets, so as to obtain a label offset map model.

4. The method as described in claim 3, characterized in that, Also includes: The tag offset mapping model is modified so that it outputs the mean and standard deviation of the tag offset distribution. Using a loss-incentive model: The modified label offset mapping model is trained to ensure that the model simultaneously outputs the confidence level of the offset predictions, where N is the total number of samples, j is the index variable, and δ j μ is the actual offset value. j To predict the expected value of the offset, σ j This represents the standard deviation of the predicted offset.

5. The method as described in claim 1, characterized in that, The urodynamic assessment model is implemented using a convolutional neural network and an expert knowledge system.

6. The method as described in claim 5, characterized in that, Also includes: Time-domain and / or frequency-domain features are acquired from multi-source urodynamic information to obtain time-domain and / or frequency-domain features; Information encoding is performed on the time-domain features and / or frequency-domain features; The features after information encoding are fused to obtain a fused feature signal. The convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and a softmax layer. The expert knowledge system includes an expert knowledge base and an inference engine. The urodynamic signal is used to extract features from the input layer, the convolutional layer, and the pooling layer to obtain at least one feature; Based on the at least one feature, the evaluation result is generated using the fully connected layer and the Softmax layer; The clinical characteristics and personal information are input into the expert knowledge system to output expert evaluation results; The intelligent assessment results are integrated with the expert assessment results to produce the final diagnostic result.

7. The method according to any one of claims 2 to 4, characterized in that, Also includes: The urodynamic signals in the training sample set are embedded in low dimension and reconstructed using a variational autoencoder, and the reconstruction error is calculated. Identify pseudo-normal samples based on a preset abnormality threshold. The pseudo-normal samples are those that are labeled as normal but have abnormal reconstruction errors. Remove pseudo-normal samples from the original training sample set; Construct a network structure with three sub-model branches to learn normal samples, abnormal samples, and pseudo-normal samples respectively; A gated network structure is constructed to receive feature information of any input sample and output weight values ​​corresponding to three sub-model branches. The weight values ​​are used to indicate which sub-model branch is more suitable to process the current sample. The branch weights output by the gated network are weighted and fused with the prediction results of the three sub-models to obtain the final prediction result. Based on the error between the sample's category label and the fusion prediction result, the training process of the urodynamic assessment model is completed by jointly optimizing the parameters of the gating network and the three sub-model branches.

8. A multimodal urodynamic assessment device, characterized in that, include: An acquisition unit is used to acquire multi-source urodynamic information, wherein the multi-source urodynamic information includes urodynamic signals, urodynamic clinical characteristics, and patient personal information; The correction unit is used to correct the urodynamic signals and urodynamic clinical features based on the patient's personal information using a label offset map model; The assessment unit is used to input the patient's personal information, as well as the corrected urodynamic signals and urodynamic clinical characteristics, into the urodynamic assessment model to obtain the patient's urodynamic assessment results.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program stored in the memory, implements the steps of the multimodal urodynamic assessment method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the multimodal urodynamic assessment method as described in any one of claims 1-7.

Images