A multi-modal data-driven intelligent monitoring method and system for pelvic floor damage

Abstract

The application provides a multi-modal data-driven pelvic floor injury intelligent monitoring method and system, and relates to the technical field of data processing. The method comprises the following steps: acquiring multiple image data and corresponding clinical text data of a person to be monitored; constructing a pelvic floor injury intelligent monitoring model; segmenting a pelvic floor region of interest in the image data; extracting image modal deep feature maps of the pelvic floor region of various image data; extracting text modal deep feature maps of various clinical text data; adaptively fusing the image modal deep feature maps and the text modal deep feature maps to obtain a fused feature map; and determining the pelvic floor injury degree of the person to be monitored according to the fused feature map. The application can realize rapid and accurate identification of pelvic floor injury and reduce the misdiagnosis rate and treatment delay.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multimodal data-driven intelligent monitoring method and system for pelvic floor injury. Background Technology

[0002] Existing diagnostic techniques for pelvic floor injuries (such as X-ray, CT, and MRI) have a high rate of missed diagnoses (30%-62%) due to bulky equipment, reliance on radiation protection, or complex operation.

[0003] Although ultrasound technology has the advantages of portability and real-time dynamic assessment, the traditional process is time-consuming and relies on the operator's experience, which is easily affected by subjective factors, resulting in low accuracy in pelvic floor injury assessment. There is an urgent need to improve its adaptability to different scenarios through intelligent transformation. Summary of the Invention

[0004] To address the technical problems of existing ultrasound technology, such as time-consuming procedures, reliance on operator experience, and susceptibility to subjective factors leading to low accuracy in pelvic floor injury assessment, this invention provides a multimodal data-driven intelligent monitoring method and system for pelvic floor injury.

[0005] The technical solutions provided by the embodiments of the present invention are as follows:

[0006] First aspect

[0007] This invention provides a multimodal data-driven intelligent monitoring method for pelvic floor injury, comprising:

[0008] S1: Acquire various image data and corresponding clinical text data of the person to be monitored;

[0009] S2: Construct an intelligent monitoring model for pelvic floor injuries;

[0010] S3: Segment the pelvic floor region of interest from the image data;

[0011] S4: Extract image modal depth feature maps of the pelvic floor region from various image data;

[0012] S5: Extract deep feature maps of text modalities from various clinical text data;

[0013] S6: Adaptively fuse the image modality depth feature map and the text modality depth feature map to obtain a fused feature map;

[0014] S7: Determine the degree of pelvic floor damage of the person to be monitored based on the fused feature map.

[0015] Second aspect

[0016] This invention provides a multimodal data-driven intelligent monitoring system for pelvic floor injury, comprising:

[0017] processor;

[0018] A memory storing computer-readable instructions, which, when executed by the processor, implement the multimodal data-driven intelligent monitoring method for pelvic floor injury as described in the first aspect.

[0019] Third aspect

[0020] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimodal data-driven intelligent monitoring method for pelvic floor injury as described in the first aspect.

[0021] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0022] In this embodiment of the invention, multimodal features are extracted using ultrasound images and clinical text multimodal data, and these features are then fused. Automated pelvic floor injury monitoring is then performed based on the fused features, eliminating the need for manual intervention. This enables rapid and accurate identification of pelvic floor injuries, reducing the rate of missed diagnoses and delays in treatment. Furthermore, ultrasound image acquisition requires only a portable ultrasound imaging device, making the process simple and convenient. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a multimodal data-driven intelligent monitoring method for pelvic floor injury provided in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of a multimodal data-driven intelligent monitoring method for pelvic floor injury provided in an embodiment of the present invention;

[0026] Figure 3 A two-dimensional ultrasound image is provided as an embodiment of the present invention.

[0027] Figure 4 This is a schematic diagram of the structure of a multimodal data-driven intelligent monitoring system for pelvic floor injury provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0029] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0030] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0031] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0032] Reference manual attached Figure 1 The diagram shows a flowchart of a multimodal data-driven intelligent monitoring method for pelvic floor injury provided by an embodiment of the present invention.

[0033] Reference manual attached Figure 2 The diagram shows a structural schematic of a multimodal data-driven intelligent monitoring method for pelvic floor injury provided by an embodiment of the present invention.

[0034] This invention provides a multimodal data-driven intelligent monitoring method for pelvic floor injury. This method can be implemented using a multimodal data-driven intelligent monitoring device for pelvic floor injury, which can be a terminal or a server. The processing flow of the multimodal data-driven intelligent monitoring method for pelvic floor injury may include the following steps:

[0035] S1: Acquire various image data and corresponding clinical text data of the person to be monitored.

[0036] Optionally, the image data includes: two-dimensional ultrasound image data, color Doppler image data, three-dimensional ultrasound image data, ultrasound contrast imaging data, and CT image data.

[0037] Reference manual attached Figure 3 This image illustrates a two-dimensional ultrasound image provided by the present invention. The ultrasound image can clearly display images of the internal and external anal sphincter muscles.

[0038] Optionally, medical image data captured by a portable device can be selected.

[0039] S2: Construct an intelligent monitoring model for pelvic floor injuries.

[0040] S3: Segment the pelvic floor region of interest from the image data.

[0041] In one possible implementation, S3 specifically involves segmenting the pelvic floor region of interest from the image data using a U-Net network.

[0042] Among them, the U-Net network is a typical convolutional neural network used for medical image segmentation, capable of accurately locating target regions and extracting contours. Its structure is symmetrically U-shaped, containing a downsampled encoder and an upsampled decoder. High-resolution feature maps are directly passed from the encoder to the decoder via skip connections, effectively fusing local details with global semantic information. This design makes U-Net perform exceptionally well when handling complex, small targets with blurred boundaries, making it particularly suitable for medical image analysis tasks requiring high-precision pixel-level segmentation.

[0043] In this embodiment of the invention, the U-Net network is used to segment the pelvic floor region of interest in the image data, which accurately delineates the region of interest for subsequent damage analysis and classification, and provides solid data and technical support for the accurate diagnosis of pelvic floor injuries on the battlefield.

[0044] In one possible implementation, S3 specifically includes sub-steps S301 to S306:

[0045] S301: The convolutional layers of the U-Net network are used to perform convolution operations on the image data to extract its features.

[0046] S302: The extracted features are downsampled through the pooling layer of the U-Net network to reduce dimensionality.

[0047] S303: Through the deconvolution layer of the U-Net network, high-level features and low-level features are fused during the upsampling operation of the extracted features.

[0048] It should be noted that traditional U-Net networks often ignore low-level features (such as edges and textures) during feature fusion due to their high noise levels. However, the pelvic floor region in medical images is usually structurally complex and morphologically diverse, with blurred boundaries with surrounding tissues. Low-level features (such as edges, textures, and contours) possess high-resolution spatial information, which helps to accurately identify the subtle anatomical boundaries between pelvic floor muscles, ligaments, and organs. Therefore, low-level features cannot be ignored in pelvic floor injury monitoring. This application proposes a novel feature fusion scheme that integrates low-level features.

[0049] Optionally, S303 specifically includes:

[0050] S3031: Perform upsampling on the extracted high-level features:

[0051]

[0052] Where c represents the upsampled high-level feature, upsample represents the upsampling operation, x1 represents the high-level feature, and W1 represents the first linear transformation convolution kernel.

[0053] S3032: Divide the upsampled high-level features into three separate channel features:

[0054]

[0055] Here, split represents the segmentation operation, and c1, c2, and c3 represent the first channel feature, the second channel feature, and the third channel feature obtained from the segmentation, respectively.

[0056] S3033: The first channel feature is fused with the low-level features to obtain the first weighted feature:

[0057]

[0058]

[0059] Where s represents the intermediate value of the attention mechanism, x2 represents the low-level feature, W2 represents the second linear transformation convolution kernel, sigmoid represents the sigmoid activation function, and y1 represents the first weighted feature.

[0060] S3034: The second channel feature is fused with the third channel feature to obtain the second weighted feature:

[0061]

[0062] Where y2 represents the second weighted feature and tanh represents the tanh activation function.

[0063] S3035: The first weighted feature and the second weighted feature are fused to obtain the final weighted feature:

[0064]

[0065] Where y represents the final weighted feature and W3 represents the third linear transformation convolution kernel.

[0066] S3036: Perform residual join concatenation operation on the final weighted features and the original high-level features:

[0067]

[0068] Where out represents the output of the deconvolution layer, and concat represents the concatenation operation.

[0069] In this embodiment of the invention, during the deconvolution stage of the U-Net network, the introduction of channel partitioning, attention mechanisms, and residual connections effectively compensates for the problem of traditional U-Net neglecting low-level features. Specifically, multiple channels are used to refine high-level features, and low-level features are selectively fused through a weighted mechanism, significantly enhancing the model's ability to perceive fine-grained structures (such as edges, textures, and contours). Simultaneously, the introduction of nonlinear activation functions such as sigmoid and tanh improves the flexibility of feature representation, and residual concatenation preserves high-level semantic information and structural continuity. This design not only improves the clarity of segmentation boundaries and the accuracy of spatial localization but also enhances the model's ability to identify complex pelvic floor structures and minor lesions, making it particularly suitable for medical image analysis tasks requiring high anatomical precision.

[0070] S304: The output of each layer is normalized through the normalization layer of the U-Net network.

[0071] S305: The classification layer of the U-Net network uses the Softmax activation function to determine the probability that each pixel in the image data is a basin bottom.

[0072] S306: Pixels with a probability greater than a preset probability are identified as the pelvic floor region through the output layer of the U-Net network.

[0073] It should be noted that those skilled in the art can set the preset probability according to actual needs, and this invention does not limit this.

[0074] In this embodiment of the invention, through the collaborative operation of the various layers of the U-Net network, not only is efficient segmentation of the pelvic floor region in images achieved, but the accuracy and robustness of the segmentation are also significantly improved. Convolutional layers extract multi-scale features, pooling layers compress information to capture global context, and deconvolutional layers restore spatial resolution and fuse features from different levels, helping to enhance the perception of details and structure. Normalization layers improve training stability and generalization ability. The classification and output layers use the Softmax mechanism to accurately distinguish each pixel, making the segmentation results more reliable. The overall process ensures that the model can stably and accurately identify the pelvic floor region in complex medical images, providing a reliable foundation for subsequent diagnosis.

[0075] S4: Extract image modal depth feature maps of the pelvic floor region from various image data.

[0076] In one possible implementation, S4 specifically involves: extracting image modal depth feature maps of the pelvic floor region from various image data using a ResNet network.

[0077] ResNet is a deep convolutional neural network designed to address the vanishing gradient problem during deep network training. Its core innovation lies in the introduction of residual connections, which incorporate input information into the output of each layer. This allows signals to bypass intermediate layers and directly propagate to subsequent layers, mitigating the vanishing or exploding gradient problem in deep networks. Through this design, ResNet maintains its depth while ensuring effective information transfer, enhancing the model's training stability and efficiency. ResNet can extract richer and more refined features, making it particularly suitable for handling complex image tasks such as medical image analysis and object recognition.

[0078] S5: Extract deep feature maps of text modalities from various clinical text data.

[0079] In one possible implementation, S5 specifically involves: extracting deep feature maps of text modalities from various clinical text data using the BERT model.

[0080] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on the Transformer architecture, designed to capture deep semantic information in text through bidirectional context learning. Unlike traditional unidirectional language models, BERT better understands the meaning of words and sentences by simultaneously considering the context of the text. It achieves excellent results in various natural language processing tasks (such as text classification and named entity recognition) by pre-training on large-scale text data and then fine-tuning it for downstream tasks.

[0081] S6: Adaptively fuse the image modality depth feature map and the text modality depth feature map to obtain a fused feature map.

[0082] In one possible implementation, an adaptive feature fusion strategy can be employed, and S6 specifically includes:

[0083] A601: Downsamples various image modal depth feature maps and text modal depth feature maps to ensure that the size of each feature map is consistent.

[0084] It should be noted that downsampling ensures that the depth feature maps of different modalities are consistent in spatial size, thus enabling effective fusion. Since image and text modalities typically have different sizes and resolutions, downsampling ensures that all feature maps are size-matched, avoiding erroneous fusion due to size differences and providing a consistent data foundation for subsequent weighting and fusion.

[0085] A602: Calculate the intermediate scores for various feature maps:

[0086]

[0087] in, Let represent the median score of the i-th feature map at position (u,v), and Softmax represent the Softmax activation function. i This represents the intermediate weight matrix of the i-th feature map. Let b represent the eigenvalue of the i-th feature map at position (u,v). i This represents the middle offset of the i-th feature map.

[0088] It's important to note that calculating intermediate scores standardizes the features at each location, allowing the contribution of each feature map to be measured within a uniform range. This not only enhances the model's sensitivity to different features but also prevents certain feature maps from excessively dominating the fusion result, ensuring a balanced contribution across different modalities.

[0089] A603: Calculate the adaptive weight coefficients for various feature maps based on their intermediate scores.

[0090]

[0091] in, represents the adaptive weight coefficient of the i-th feature map at position (u,v), and n represents the total number of feature map categories.

[0092] It's important to note that by calculating adaptive weight coefficients based on the median score of each feature map, the influence of different feature maps can be dynamically adjusted according to their performance at specific locations. The advantage of this adaptive mechanism is that it can dynamically adjust the weighting strategy based on the feature quality at each location (such as the precision of key regions or boundaries), thereby enhancing the model's ability to focus on specific regions and self-adjusting based on the performance of each feature map, ensuring that more important features receive greater attention.

[0093] A604: Based on an adaptive feature fusion strategy, the deep features of image data are adaptively fused according to the adaptive weight coefficients of various feature maps to obtain a fused feature map.

[0094]

[0095] Where A represents the fused feature map, A uv This represents the fused feature value at position (u, v) in the fused feature map.

[0096] It should be noted that the contribution of each modality to the final fusion result can be precisely controlled according to the weight of each feature map. This ensures the enhancement of important features and the suppression of irrelevant features, thereby improving the expressiveness of the fused feature map in the task. In multimodal fusion tasks, this adaptive weighting helps the model to more intelligently integrate information from different sources, enhancing the model's expressive power.

[0097] In this embodiment of the invention, an adaptive feature fusion strategy is employed to intelligently adjust the weights of feature maps from different modalities, thereby more effectively fusing deep features from image and text modalities. In this strategy, each feature map is downsampled and intermediate scores are calculated, and then adaptive weight coefficients are calculated based on these scores. This allows the network to automatically adjust the contribution of different features according to the feature importance at each location. This adaptive weighting mechanism enhances the model's focus on valuable information and suppresses redundant or noisy features, thereby improving the expressive power of the fused feature maps. The final fused features more accurately capture key patterns in image and text information, thus improving the model's performance and robustness in complex tasks.

[0098] However, in multimodal data, different modalities may provide conflicting information (e.g., images and text may lead to different conclusions). Adaptive feature fusion mainly relies on weight calculation for fusion, but if the features of a certain modality have significant bias or inaccuracy, a simple weighting strategy may further amplify the error of the fusion result, failing to effectively resolve the conflict problem. To address this, this application provides a feature fusion method that can effectively handle conflicting and inconsistent information between different modalities: a DS-based evidence fusion strategy.

[0099] In one possible implementation, S6 specifically includes:

[0100] B601: Mapping each deep feature map to an evidence vector:

[0101]

[0102]

[0103] Among them, e img W represents the evidence vector corresponding to the image modality depth feature map. img F represents the image modality weight mapping parameter. img Represents the image modality depth feature map, bimg Represents the image modality bias mapping parameters, and ReLU represents the ReLU activation function. e represents the strength of evidence for the i-th classification category in the image modality depth feature map. txt W represents the evidence vector corresponding to the deep feature map of the text modality. txt F represents the text modality weight mapping parameter. txt Represents the deep feature map of text modalities, b txt This represents the text modality bias mapping parameters. represents the evidence strength of the text modality deep feature map for the i-th classification category, and n represents the total number of classification categories.

[0104] It's important to note that converting the deep feature maps of image and text modalities into evidence vectors using the ReLU activation function and weight mapping transforms the feature information of each modality into support for different classification categories (evidence strength). This mapping not only preserves the semantic information of each feature map but also introduces non-linear features through the ReLU activation function, enhancing the model's ability to learn complex patterns. This step helps extract more discriminative features from different modalities, providing a strong semantic evidence foundation for subsequent fusion.

[0105] B602: Calculate the uncertainty of each depth feature map based on the evidence vector:

[0106]

[0107]

[0108] Among them, u img This indicates the uncertainty of image modal depth feature maps in detecting pelvic floor injuries, u txt This represents the uncertainty of the text modality depth feature map when performing pelvic floor injury detection, and d represents the balance constant used to adjust the uncertainty sensitivity.

[0109] It should be noted that uncertainty is calculated by normalizing the sum of the evidence vectors and adding a balance constant. This operation assigns an uncertainty measure to each feature map. In pelvic floor injury monitoring tasks, different modalities (such as images and text) may provide incomplete or conflicting information in certain situations. Introducing uncertainty helps measure the credibility of different modalities, especially when the data quality is poor or there is a lot of noise. This can improve the stability of the model and avoid over-reliance on unreliable information.

[0110] B603: Based on the evidence vector, calculate the probability estimate of each deep feature map for each classification category:

[0111]

[0112]

[0113] in, This represents the probability estimate of the image modal depth feature map for the i-th classification category. This represents the probability estimate of the text modality deep feature map for the i-th classification category.

[0114] It's important to note that by transforming the evidence vector into probability estimates, this step provides an accurate probability assessment for each classification category. This not only quantifies the contribution of each modality to a particular classification but also helps the model handle the weight allocation between modalities. In the scenario of pelvic floor injury monitoring, these probability estimates help clarify the role of each modality in determining the classification and ensure that the contribution of each modality to the final decision is transparent and evidence-based.

[0115] B604: Calculate the conflict coefficient based on uncertainty and probability estimates:

[0116]

[0117] in, This represents the conflict coefficient between the image modality and the text modality. This represents the probability estimate of the text modality deep feature map for the j-th classification category.

[0118] Compared to traditional DS evidence theory, incorporating the uncertainties of each modality in pelvic floor injury detection when calculating the conflict coefficient effectively identifies and quantifies information conflicts between image and text modalities. Since image and text modalities may provide different predictions, the conflict coefficient, by considering the uncertainties of each modality, adjusts the relative contributions between different modalities, reducing decision bias caused by modal inconsistencies or error accumulation. In this way, the model can more intelligently adjust its processing of conflicting information, ensuring a more accurate final output.

[0119] B605: Based on the DS evidence fusion strategy, the trust distribution for each classification category is calculated according to the conflict coefficient.

[0120]

[0121]

[0122] in, This represents the support of the image modality depth feature map for the i-th classification category. This represents the support of the text modality deep feature map for the i-th classification category.

[0123] It's important to note that calculating the trust distribution for each modality to each classification category is a step that, by appropriately adjusting the support levels of the image and text modalities, ensures that the fused information is more reliable and authoritative. In the pelvic floor injury monitoring task, this trust distribution provides a quantified trust value for each category, helping the model dynamically adjust its decision-making process based on the reliability of information from different modalities, thereby improving the credibility and accuracy of predictions.

[0124] B606: Calculate the fusion support for each classification category based on the trust distribution:

[0125]

[0126] in, This represents the fusion support of the evidence for the i-th classification category.

[0127] It should be noted that the fusion support is calculated by multiplying the trust distributions of the image modality and the text modality. This operation helps to integrate the support information from different modalities to obtain a more accurate fusion decision. Through this strategy, the effective information from the image and text modalities can work synergistically, allowing the final fused feature map to better reflect the information from both, enhancing the model's ability to recognize each category, especially when there are significant modal differences.

[0128] B607: Generate a fusion feature map based on the fusion support:

[0129]

[0130] Where A represents the fused feature map, W f Let b represent the weight matrix of the fusion layer. f This indicates the fusion layer bias term.

[0131] It should be noted that the final fusion feature map is generated based on the calculated fusion support. This process, through linear transformation of weights and biases combined with the fusion support, effectively integrates the deep features of both image and text modalities. The generated fusion feature map can simultaneously fuse information from different modalities, fully leveraging the strengths of each modality, thereby providing more accurate decision support in pelvic floor injury monitoring. This fusion feature map provides a more comprehensive and efficient input for subsequent classification tasks, enhancing the model's ability to handle complex tasks.

[0132] In this embodiment of the invention, by employing a DS-based evidence fusion strategy, the method can flexibly process deep feature maps from image and text modalities, and dynamically adjust the contribution of each modality to the classification results by introducing evidence vectors, conflict coefficients, and trust distributions. First, calculating the evidence vector and assessing uncertainty quantifies the reliability of each modality. Then, based on uncertainty and probability estimation, fusing information from each modality effectively avoids conflicts between modalities and enhances the focus on reliable features according to the trust distribution. Finally, by generating feature maps through support fusion, the model can intelligently integrate information from different modalities, improving the accuracy and robustness of identifying fine-grained features in complex tasks (such as pelvic floor injury monitoring), especially providing a more robust and reliable decision-making basis when modal data quality is inconsistent.

[0133] S7: Determine the degree of pelvic floor damage in the monitored individuals based on the fusion feature map.

[0134] Optionally, based on the fused feature map, the Softmax activation function is used to determine the degree of pelvic floor damage in the person being monitored. The use of the Softmax activation function is a very mature existing technology, and will not be described in detail here.

[0135] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0136] In this embodiment of the invention, multimodal features are extracted using ultrasound images and clinical text multimodal data, and these features are then fused. Automated pelvic floor injury monitoring is then performed based on the fused features, eliminating the need for manual intervention. This enables rapid and accurate identification of pelvic floor injuries, reducing the rate of missed diagnoses and delays in treatment. Furthermore, ultrasound image acquisition requires only a portable ultrasound imaging device, making the process simple and convenient.

[0137] Because the pelvic floor loss monitoring model provided in this embodiment of the invention is designed for a specific environment, it faces the challenge of missing training data during the training process. The number of training samples is limited, and the generated training samples (especially ultrasound images after pelvic floor injury perfusion) are prone to physiological distortion. To address this, this embodiment of the invention proposes a novel image generation framework that uses source images and text information to generate ultrasound images after pelvic floor injury perfusion.

[0138] First, based on the physical characteristics of ultrasonic wave attenuation, a vertical axis dynamic noise control strategy is designed (weak noise at the top of the image and strong noise at the bottom). The noise in the image generation process is decomposed into the superposition of anatomical attenuation noise (vertical gradient of the sound wave) and battlefield degradation noise (electromagnetic interference / motion artifacts).

[0139]

[0140] in, Denotes the dynamic noise scheduling function, x t The image at time t represents the vertical coordinate, t represents time, and α represents the vertical noise intensity coefficient. λ represents the vertical attenuation factor, e represents the natural constant, λ represents the attenuation rate coefficient, γ represents the electromagnetic interference noise weighting coefficient, and N represents the vertical attenuation factor. EM The field represents the electronic interference noise field, δ represents the motion artifact noise weighting coefficient, and N represents the electronic interference noise field. mot This represents motion artifact noise.

[0141] In this embodiment of the invention, a vertical axis noise control strategy is designed by simulating the attenuation characteristics of ultrasound waves in tissues, making the noise at the bottom of the image stronger and the noise at the top weaker, which conforms to the noise distribution law of actual ultrasound images.

[0142] Based on the actual situation of ultrasound imaging, the standard forward propagation was modified by introducing a method that... The proposed forward distribution is centered on noise and integrates it in a way that better aligns with ultrasound image generation. Noise is introduced by multiplying the current noise scheduler, the previous time-in image, and the current time-in scheduler point-by-point.

[0143]

[0144]

[0145] Where q represents the conditional probability distribution, x t-1 This represents the graph at time t-1, where N represents a normal distribution. This represents the accumulated noise intensity factor over the first t time steps. Let I represent the noise scheduler accumulated over the first t time steps, and let I represent the unit covariance matrix.

[0146] It should be noted that this method guides the behavior of the diffusion model by simulating the gradual decay of echo intensity, introducing more noise at the bottom of the image during the forward process. Therefore, it converges to a standard Gaussian distribution faster at the bottom of the image than at the top.

[0147] The posterior distribution of the newly derived physics-inspired diffusion model is determined as follows:

[0148]

[0149] Where q represents the conditional probability distribution. c represents the learned mean function. θ This represents the learned covariance function.

[0150] Secondly, based on the dynamic noise control strategy, a generative adversarial diffusion framework is further constructed. In each step of the reverse process, the target image x0 and the source image y are explicitly introduced as dissection priors to guide the process, and the real image is gradually approximated through recursive optimization.

[0151]

[0152] in, Let G represent the target image estimated in the (r+1)th iteration. θ Let θ represent the image generator, and let θ represent the model parameters of the image generator. Let R represent the target image estimated in the r-th iteration, where R represents the total number of iterations.

[0153] Furthermore, the generator is trained using pixel-level L1 and adversarial loss terms via gradient descent.

[0154] In this embodiment of the invention, generative adversarial networks are combined with diffusion models. Through recursive optimization steps, the network gradually approximates the real image. Anatomical prior knowledge (source image and target image) is added as guidance at each step of image generation, which further improves the anatomical rationality and realism of the generated image.

[0155] Reference manual attached Figure 4 The diagram shows a structural schematic of a multimodal data-driven intelligent monitoring system for pelvic floor injury provided by the present invention.

[0156] The present invention also provides a multimodal data-driven intelligent monitoring system 20 for pelvic floor injury, applied to the above-mentioned multimodal data-driven intelligent monitoring method for pelvic floor injury, comprising:

[0157] Processor 201.

[0158] The memory 202 stores computer-readable instructions, which, when executed by the processor 201, implement the multimodal data-driven intelligent monitoring method for pelvic floor injury as described in the method embodiment.

[0159] The multimodal data-driven intelligent monitoring system 20 for pelvic floor injury provided by this invention can execute the above-described multimodal data-driven intelligent monitoring method for pelvic floor injury and achieve the same or similar technical effects. To avoid duplication, this invention will not elaborate further.

[0160] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0161] In this embodiment of the invention, multimodal features are extracted using ultrasound images and clinical text multimodal data, and these features are then fused. Automated pelvic floor injury monitoring is then performed based on the fused features, eliminating the need for manual intervention. This enables rapid and accurate identification of pelvic floor injuries, reducing the rate of missed diagnoses and delays in treatment. Furthermore, ultrasound image acquisition requires only a portable ultrasound imaging device, making the process simple and convenient.

[0162] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0163] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0164] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0165] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0166] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0167] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0168] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0169] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0170] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0171] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0173] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] This invention provides a computer-readable storage medium storing a computer program thereon, characterized in that, when executed by a processor, the program implements a multimodal data-driven intelligent monitoring method for pelvic floor injury as described in the method embodiment.

[0175] The present invention provides a computer-readable storage medium that can implement the steps and effects of the multimodal data-driven intelligent monitoring method for pelvic floor injury described in the above-described method embodiments. To avoid repetition, the present invention will not elaborate further.

[0176] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0177] The following points need to be explained:

[0178] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0179] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0180] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0181] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-modal data-driven intelligent monitoring method for pelvic floor damage, characterized in that, The method comprises the following steps: S1: obtaining multiple image data and corresponding clinical text data of a person to be monitored; S2: constructing an intelligent monitoring model for pelvic floor injury; S3: segmenting a pelvic floor region of interest in the image data; S4: extracting image modality deep feature maps of the pelvic floor region in various image data; S5: extracting text modality deep feature maps of various clinical text data; S6: adaptively fusing the image modality deep feature maps and the text modality deep feature maps to obtain a fused feature map; S7: determining the degree of pelvic floor injury of the person to be monitored according to the fused feature map; S6 specifically comprises: S601: mapping each deep feature map into an evidence vector; ； ； wherein e img denotes the evidence vector corresponding to the image modality deep feature map, W img denotes the image modality weight mapping parameter, F img denotes the image modality deep feature map, b img denotes the image modality bias mapping parameter, ReLU denotes the ReLU activation function, denotes the evidence strength of the image modality deep feature map on the i-th classification category, e txt denotes the evidence vector corresponding to the text modality deep feature map, W txt denotes the text modality weight mapping parameter, F txt denotes the text modality deep feature map, b txt denotes the text modality bias mapping parameter, denotes the evidence strength of the text modality deep feature map on the i-th classification category, n denotes the total number of classification categories; B602: calculating the uncertainty of each deep feature map according to the evidence vector; ； ； wherein u img represents the uncertainty of the image modality deep feature map when conducting pelvic floor injury detection monitoring, u txt represents the uncertainty of the text modality deep feature map when conducting pelvic floor injury detection monitoring, d represents a balance constant for regulating the uncertainty sensitivity; B603: calculating the probability estimate of each deep feature map for each classification category according to the evidence vector; ； ； wherein, denotes the probability estimate of the image modality deep feature map for the i-th classification class, denotes the probability estimate of the text modality deep feature map for the i-th classification class; B604: calculating a conflict coefficient according to the uncertainty and the probability estimate; ； wherein, denotes a conflict coefficient between the image modality and the text modality, denotes a probability estimate of the text modality deep feature map for the j-th classification class; B605: calculating a trust distribution for each classification category according to the conflict coefficient based on a D-S evidence fusion strategy; ； ； wherein, represents the support of the image modality deep feature map to the i-th classification category, represents the support of the text modality deep feature map to the i-th classification category; B606: calculating a fused support degree for each classification category according to the trust distribution; ； wherein, represents the fusion support degree of the evidence for the i-th classification category; B607: generating a fused feature map according to the fused support degree; ； Wherein, A represents the fusion feature map, W f represents the fusion layer weight matrix, b f represents the fusion layer bias term.

2. The multi-modal data driven intelligent monitoring of pelvic floor impairments method as claimed in claim 1, wherein, The image data comprises two-dimensional ultrasound image data, color Doppler image data, three-dimensional ultrasound image data, ultrasound contrast image data, and CT image data.

3. The multi-modal data driven intelligent monitoring of pelvic floor impairments method as claimed in claim 1, wherein, S3 specifically comprises segmenting the pelvic floor region of interest in the image data through a U-Net network.

4. The multi-modal data driven intelligent monitoring of pelvic floor impairments method as claimed in claim 1, wherein, S3 specifically comprises: S301: performing convolution operation on the image data through the convolution layer of the U-Net network to extract features of the image data; S302: performing down-sampling operation on the extracted features through the pooling layer of the U-Net network to perform dimension reduction processing; S303: performing fusion of high-level features and low-level features in the process of up-sampling operation on the extracted features through the deconvolution layer of the U-Net network; S304: performing normalization processing on the output of each layer through the normalization layer of the U-Net network; S305: determining the probability of each pixel in the image data being a pelvic floor through the classification layer of the U-Net network through a Softmax activation function; S306: determining the pixels with a probability greater than a preset probability as the pelvic floor region through the output layer of the U-Net network.

5. The multi-modal data driven intelligent monitoring of pelvic floor impairments method as claimed in claim 4, wherein, S303 specifically comprises: S3031: performing up-sampling operation on the extracted high-level features; S3032: equally dividing the up-sampled high-level features into three separate channel features; S3033: fusing the first channel feature with the low-level features to obtain a first weighted feature; S3034: fusing the second channel feature with the third channel feature to obtain a second weighted feature; S3035: fusing the first weighted feature and the second weighted feature to obtain a final weighted feature; S3036: performing residual connection splicing operation on the final weighted feature and the original high-level feature.

6. The multi-modal data driven intelligent monitoring of pelvic floor impairments method as claimed in claim 1, wherein, The S4 is specifically: Through the ResNet network, the image modal deep feature map of the pelvic floor region of various image data is extracted.

7. The multi-modal data driven intelligent monitoring of pelvic floor impairments method of claim 1, wherein, The S5 is specifically: Through the BERT model, the text modal deep feature map of various clinical text data is extracted.

8. A multi-modal data-driven intelligent monitoring system for pelvic floor impairments, the system comprising: Include: A processor; A memory, the memory has computer readable instructions stored thereon, the computer readable instructions are executed by the processor, and the multi-modal data driven pelvic floor damage intelligent monitoring method is realized as any one of claims 1-7.

Images