Brain injury prediction method, device and equipment based on newborn craniocerebral ultrasound image and medium

Abstract

The invention discloses a brain injury prediction method and device based on a newborn craniocerebral ultrasound image, equipment and a medium. The method comprises the following steps: acquiring a newborn craniocerebral ultrasound video; according to the newborn craniocerebral ultrasonic video, determining an anatomical structure prediction result of each newborn craniocerebral ultrasonic video frame in the newborn craniocerebral ultrasonic video through the anatomical structure detection module; according to an anatomical structure prediction result of each newborn craniocerebral ultrasonic video frame, extracting a standard section image from the newborn craniocerebral ultrasonic video through the standard section extraction module; and according to all the extracted standard section images, outputting a predicted brain injury category through the brain injury prediction model. According to the method, standard section image extraction and brain injury category prediction are combined, and brain injury category prediction can be carried out based on the newborn craniocerebral ultrasonic video, so that a doctor does not need to manually select the section image, and the brain injury category prediction efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of biomedical technology, and in particular to a method, device, equipment and medium for predicting brain injury based on neonatal cranial ultrasound images. Background Technology

[0002] Currently, neonatal craniocerebral injury is a significant cause of neurodevelopmental abnormalities in premature and low birth weight infants, potentially leading to cerebral palsy, cognitive impairment, and other lifelong neurological disorders. Due to its non-invasive, portable, and low-cost advantages, cranial ultrasound is widely used in neonatal intensive care for initial screening and follow-up of brain injuries. However, current neonatal cranial ultrasound examinations rely heavily on the operator's expertise and clinical experience for both image acquisition and diagnostic interpretation, resulting in significant subjectivity. This subjectivity leads to low standardization in image acquisition, inconsistent diagnostic processes, and low efficiency, making it difficult to meet the clinical needs of large-scale, high-quality screening. Furthermore, the identification of standard sections and lesion assessment requires manual frame-by-frame analysis by professional radiologists, a cumbersome and time-consuming process with inherent subjectivity and inconsistent results, limiting the widespread application of ultrasound in primary healthcare institutions.

[0003] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a method, device, equipment and medium for predicting brain injury based on neonatal cranial ultrasound images, in order to address the shortcomings of the existing technology.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a method for predicting brain injury based on neonatal cranial ultrasound images. This method utilizes a trained brain injury prediction model, which includes a slice extraction model and a brain injury prediction model. The slice extraction model includes an anatomical structure detection module and a standard section extraction module. Specifically, the method for predicting brain injury based on neonatal cranial ultrasound images includes:

[0006] Obtaining ultrasound video of the newborn's brain;

[0007] Based on the neonatal cranial ultrasound video, the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video is determined by the anatomical structure detection module.

[0008] Based on the anatomical structure prediction results of each frame of neonatal cranial ultrasound video, standard section images are extracted from the neonatal cranial ultrasound video using the standard section extraction module.

[0009] Based on all extracted standard cross-sectional images, the brain injury prediction model outputs a predicted brain injury category.

[0010] The brain injury prediction method based on neonatal cranial ultrasound images, wherein the step of extracting standard section images from the neonatal cranial ultrasound video using the standard section extraction module based on the anatomical structure prediction results of each frame of neonatal cranial ultrasound video specifically includes:

[0011] The predicted score for each neonatal cranial ultrasound video frame is determined based on the confidence level of the anatomical structure category of each frame.

[0012] The neonatal cranial ultrasound video is divided into several video frame clusters based on the predicted score and anatomical structure prediction category, wherein each video frame cluster corresponds to a standard section.

[0013] In each video frame cluster, a video frame is selected according to the predicted score, and this video frame is used as the standard cross-sectional image corresponding to the video frame cluster.

[0014] The brain injury prediction method based on neonatal cranial ultrasound images, wherein the anatomical structure prediction result includes anatomical structure category confidence; the determination of the prediction score for each neonatal cranial ultrasound video frame based on the anatomical structure category confidence of each frame specifically includes:

[0015] The preset weights corresponding to each frame of neonatal cranial ultrasound video are determined based on the predicted anatomical structure category.

[0016] The predicted score for each frame of neonatal cranial ultrasound video is calculated based on the confidence level of the anatomical structure category and the preset weight.

[0017] The brain injury prediction method based on neonatal cranial ultrasound images, wherein dividing the neonatal cranial ultrasound video into several video frame clusters according to the prediction score and anatomical structure prediction category specifically includes:

[0018] Read the frame number of each neonatal cranial ultrasound video frame, and construct two-dimensional input data for each neonatal cranial ultrasound video frame based on the frame number and the predicted score;

[0019] Based on the anatomical structure prediction category, the neonatal cranial ultrasound video is clustered according to the two-dimensional input data to obtain several video frame clusters. Among them, the neonatal cranial ultrasound video frames in each video frame cluster have the same anatomical structure prediction category, are continuous in time, and have a prediction score higher than a preset score threshold.

[0020] The brain injury prediction method based on neonatal cranial ultrasound images, wherein the training process of the slice extraction model specifically includes:

[0021] We acquired cranial ultrasound video data from multiple neonatal cases. Each cranial ultrasound video data set included several cranial ultrasound images, and each cranial ultrasound image carried anatomical structure annotations.

[0022] Cranial ultrasound video data pairs were selected from multiple neonatal cases, where the two cranial ultrasound video data pairs had the same number of standard sections.

[0023] The standard section images in each selected pair of cranial ultrasound video data are linearly blended to obtain a mixed ultrasound image;

[0024] Based on the cranial ultrasound video data and all mixed ultrasound images of the multiple neonatal cases, a preset slice extraction model was trained to obtain a trained slice extraction model.

[0025] The brain injury prediction method based on neonatal cranial ultrasound images, wherein the step of training a preset slice extraction model to obtain a trained slice extraction model based on cranial ultrasound video data from multiple neonatal cases and the mixed ultrasound images specifically includes:

[0026] For each cranial ultrasound image in each cranial ultrasound video data, a target mixed ultrasound image corresponding to the cranial ultrasound image is found among all mixed ultrasound images, wherein the target mixed ultrasound image is generated based on the cranial ultrasound image.

[0027] For the cranial ultrasound image where the target mixed ultrasound image is found, the preset slice extraction model is trained by combining the cranial ultrasound image and the target mixed ultrasound image;

[0028] For cranial ultrasound images where no target mixed ultrasound image was found, a preset slice extraction model was trained based on the cranial ultrasound image.

[0029] The loss function for training the preset slice extraction model using the combined cranial ultrasound image and its corresponding hybrid ultrasound image is:

[0030]

[0031] in, Let α represent the loss function, and p represent the weighting coefficients. A Represents a cranial ultrasound image, p mixed Represents a mixed ultrasound image, y A This represents the anatomical annotation results of a cranial ultrasound image, yB This represents the anatomical annotation results of another cranial ultrasound image, which is then mixed with the aforementioned cranial ultrasound image to generate the hybrid ultrasound image. Indicates based on y A ,y A ;p a ,p mixed The constructed loss function, Based on y A ,y B ;p A ,p mixed The constructed loss function.

[0032] The brain injury prediction method based on neonatal cranial ultrasound images, wherein the step of outputting a predicted brain injury category through the brain injury prediction model based on all extracted standard cross-sectional images specifically includes:

[0033] All extracted standard cross-sectional images are input into the brain injury prediction model, and the brain injury prediction model determines the image-level predicted brain injury category corresponding to each standard cross-sectional image.

[0034] Based on the image-level prediction of brain injury categories corresponding to all standard cross-sectional images, the case-level predicted brain injury category is determined by the brain injury prediction model.

[0035] The second aspect of this application provides a brain injury prediction device based on neonatal cranial ultrasound images, which applies a trained brain injury prediction model. The brain injury prediction model includes a slice extraction model and a brain injury prediction model. The slice extraction model includes an anatomical structure detection module and a standard section extraction module. Specifically, the brain injury prediction device based on neonatal cranial ultrasound images includes:

[0036] The acquisition module is used to acquire ultrasound videos of the newborn's cranium;

[0037] The first control module is used to determine the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video through the anatomical structure detection module based on the neonatal cranial ultrasound video; and to extract standard section images from the neonatal cranial ultrasound video through the standard section extraction module based on the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video.

[0038] The second control module is used to output a predicted brain injury category based on all extracted standard cross-sectional images through the brain injury prediction model.

[0039] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the brain injury prediction method based on neonatal cranial ultrasound images as described above.

[0040] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory;

[0041] The memory stores a computer-readable program that can be executed by the processor;

[0042] When the processor executes the computer-readable program, it implements the steps in the brain injury prediction method based on neonatal cranial ultrasound images as described above.

[0043] Beneficial Effects: Compared with existing technologies, this application provides a method, apparatus, device, and medium for predicting brain injury based on neonatal cranial ultrasound images. The method includes acquiring neonatal cranial ultrasound video; determining the anatomical structure prediction result for each frame of the neonatal cranial ultrasound video using an anatomical structure detection module; extracting standard section images from the neonatal cranial ultrasound video using a standard section extraction module based on the anatomical structure prediction results for each frame; and outputting a predicted brain injury category based on all extracted standard section images using a brain injury prediction model. This application combines standard section image extraction and brain injury category prediction, enabling brain injury category prediction based on neonatal cranial ultrasound video. This eliminates the need for doctors to manually select section images, improving the efficiency of brain injury category prediction. Furthermore, using multiple standard section images for brain injury category prediction provides richer semantic information, thereby improving the accuracy of brain injury category prediction. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart of a brain injury prediction method based on neonatal cranial ultrasound images provided in this application embodiment.

[0046] Figure 2 A flowchart illustrating the principle of the brain injury prediction method based on neonatal cranial ultrasound images provided in this application embodiment.

[0047] Figure 3 A schematic diagram of the brain injury prediction device based on neonatal cranial ultrasound images provided in this application embodiment.

[0048] Figure 4 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0049] This application provides a method, apparatus, device, and medium for predicting brain injury based on neonatal cranial ultrasound images. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description, with reference to the accompanying drawings and embodiments, further illustrates this application. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0051] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0052] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0053] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0054] This embodiment provides a brain injury prediction method based on neonatal cranial ultrasound images. It applies a trained brain injury prediction model, which includes a slice extraction model and a brain injury prediction model. The slice extraction model extracts standard section images from neonatal cranial ultrasound videos, and the brain injury prediction model predicts brain injury based on these extracted standard section images. The slice extraction model includes an anatomical structure detection module and a standard section extraction module. The anatomical structure detection module detects anatomical structures in the neonatal cranial ultrasound video frames, and the standard section extraction module determines all standard section images in the neonatal cranial ultrasound video based on the anatomical structure detection results. The brain injury prediction model can extract standard cross-sectional images from the neonatal cranial ultrasound video based on the anatomical structure prediction results of each frame of the neonatal cranial ultrasound video. Then, it can predict the brain injury category based on all extracted standard cross-sectional images (one or more). This combination of standard cross-sectional image extraction and brain injury category prediction not only allows for the extraction of standard cross-sectional images from the neonatal cranial ultrasound video, but also enables the prediction of brain injury category using multiple extracted standard cross-sectional images. This provides richer semantic information for brain injury category prediction and improves the accuracy and efficiency of brain injury category prediction.

[0055] like Figure 1 and Figure 2 As shown in the embodiments of this application, the brain injury prediction method based on neonatal cranial ultrasound images specifically includes:

[0056] S10. Obtain ultrasound video of the newborn's cranium.

[0057] Specifically, the neonatal cranial ultrasound video includes multiple ultrasound video frames, that is, multiple ultrasound video images. The neonatal cranial ultrasound video can be multiple ultrasound images acquired during a cranial ultrasound examination of a newborn. The neonatal cranial ultrasound video can be acquired in real time by an ultrasound device, stored locally on an electronic device running the brain injury prediction method based on neonatal cranial ultrasound images provided in this application, or transmitted by an external device, etc.

[0058] Furthermore, after acquiring the neonatal cranial ultrasound video, to improve the image quality of each frame, the neonatal cranial ultrasound video can be standardized. This standardization process involves converting the neonatal cranial ultrasound video frames into grayscale images and binarizing them (e.g., using the Otsu adaptive thresholding method) to distinguish the effective imaging region from the black background. Subsequently, a contour detection algorithm is used to identify all connected regions within the effective imaging region. Effective contours that meet the characteristics of the effective ultrasound imaging region are selected by setting a relative area threshold (e.g., not less than 15% of the total image pixels) and an aspect ratio constraint (e.g., between 0.5 and 2.5). Finally, the image region corresponding to the largest effective contour is selected as the main imaging region. The neonatal cranial ultrasound video frame is then cropped based on the main imaging region to obtain the region of interest (ROI), which is used as the standardized neonatal cranial ultrasound video frame.

[0059] S20. Based on the neonatal cranial ultrasound video, the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video is determined by the anatomical structure detection module.

[0060] Specifically, the anatomical structure prediction results include the predicted anatomical structure category, the confidence level of the predicted anatomical structure category, and the predicted anatomical structure bounding box. The anatomical structure refers to the anatomical structures included in the standard cross-section, including the anterior horn of the lateral ventricle, the thalamus and basal ganglia, the corpus callosum, the third ventricle, the body of the lateral ventricle, and the cerebellum. Therefore, the anatomical structure category includes the anterior horn of the lateral ventricle, the thalamus and basal ganglia, the corpus callosum, the third ventricle, the body of the lateral ventricle, and the cerebellum. Thus, the anatomical structure prediction category is the anterior horn of the lateral ventricle, the thalamus and basal ganglia, the corpus callosum, the third ventricle, the body of the lateral ventricle, and the cerebellum. The confidence level of the anatomical structure category is the degree of confidence of the predicted anatomical structure category. The anatomical structure prediction results may also include the predicted anatomical structure bounding box, which can be represented as (x, y, w, h, θ), where x and y represent the coordinates of the center of the rotated box, w and h represent the width and height of the box, and θ represents the rotation angle.

[0061] In one embodiment, the anatomical structure detection module may include an encoding unit and a decoding unit. The encoding unit encodes neonatal cranial ultrasound video frames to obtain encoded features, and the decoding unit decodes based on the encoded features to obtain anatomical structure prediction results for the neonatal cranial ultrasound video frames. The encoding unit may include a backbone network, a multi-layer stacked Transformer encoder, and a fusion network. The backbone network extracts features from the neonatal cranial ultrasound video frames to obtain multi-scale features, which include image features extracted from the last three layers of the backbone network, denoted as (C3, C4, C5). The Transformer encoder determines semantic enhancement features based on the last layer feature C5. The semantic enhancement feature acquisition process involves reducing the dimensionality of the last layer feature C5 to a uniform number of channels using a 1×1 convolution, then superimposing a learnable two-dimensional sinusoidal positional code to obtain image features with positional codes that retain spatial structural information. These positional code-encoded image features are then flattened and fed into a multi-layer stacked Transformer encoder (e.g., a 3-layer stacked Transformer encoder). Long-distance dependency modeling and semantic enhancement are performed by the Transformer encoder to obtain semantic enhancement features. The fusion network is used to fuse the second-to-last layer features C4 and the third-to-last layer features C3 with the semantic enhancement features to obtain the encoded features. Specifically, the fusion process involves first merging the second-to-last layer features C4 and C3 (passed through a 1×1 convolution) with the semantic enhancement features to have the same number of channels. Then, the second-to-last layer features C4 and C3 with the same number of channels, along with the semantic enhancement features, are fused across scales through a fusion block to obtain the encoded features. This fusion block may include a 1×1 convolutional layer, multiple reparameterized convolutional blocks (RepBlocks), and element-wise addition operations between features.

[0062] Furthermore, the decoding unit is used to perform cross-attention interaction between the initialized learnable target query vector and the encoded features to determine the anatomical structure prediction result. The learnable target query vector includes positional embedding components and semantic embedding components. The decoding unit includes a multi-layered stacked Transformer decoder. The target query vector and encoded features are input into the multi-layered stacked Transformer decoder, and cross-attention interaction is performed through the multi-layered stacked Transformer decoder, combined with a self-attention mechanism to achieve information exchange between queries.

[0063] S30. Based on the anatomical structure prediction results of each frame of neonatal cranial ultrasound video, extract standard section images from the neonatal cranial ultrasound video using the standard section extraction module.

[0064] Specifically, the standard section image is a single frame from a neonatal cranial ultrasound video. In other words, the standard section extraction module selects a video frame from the neonatal cranial ultrasound video as the standard section. The standard section image can be one or more of the three standard sections in the coronal plane (anterior horn of the lateral ventricle, the third ventricle, and the lateral ventricle positional section) and the three standard sections in the sagittal plane (median sagittal, left parasagittal, and right parasagittal).

[0065] Furthermore, since the anatomical structures included in each of the three standard sections in the coronal plane and the three standard sections in the sagittal plane are different, after detecting the anatomical structure prediction results of each neonatal cranial ultrasound video frame, the neonatal cranial ultrasound video frames can be grouped according to the correspondence between the anatomical structures and the standard sections. Then, the final neonatal cranial ultrasound video frame is selected from the neonatal cranial ultrasound video frame group corresponding to each standard section, and the standard section image is selected in the neonatal cranial ultrasound video frame according to the anatomical structure prediction box corresponding to the neonatal cranial ultrasound video frame.

[0066] Based on this, the step of extracting standard section images from the neonatal cranial ultrasound video using the standard section extraction module, based on the anatomical structure prediction results of each frame of neonatal cranial ultrasound video, specifically includes:

[0067] S31. Determine the predicted score for each frame of neonatal cranial ultrasound video based on the confidence level of the anatomical structure category of each frame.

[0068] S32. The neonatal cranial ultrasound video is divided into several video frame clusters according to the predicted score and anatomical structure prediction category, wherein each video frame cluster in the several video frame clusters corresponds to a standard section.

[0069] S33. Select one video frame from each video frame cluster according to the predicted score, and use this video frame as the standard cross-sectional image corresponding to the video frame cluster.

[0070] In step S31, the anatomical structure prediction results include anatomical structure category confidence scores. Each neonatal cranial ultrasound video frame is quantitatively scored based on its anatomical structure category confidence score to determine its predicted view as a standard section. The prediction score indicates the probability that the neonatal cranial ultrasound video is a standard section; a higher prediction score indicates a higher probability that the neonatal cranial ultrasound video is a standard section, and vice versa.

[0071] In one embodiment, determining the predicted score for each frame of neonatal cranial ultrasound video based on the confidence level of the anatomical structure category of each frame specifically includes:

[0072] The preset weights corresponding to each frame of neonatal cranial ultrasound video are determined based on the predicted anatomical structure category.

[0073] The predicted score for each frame of neonatal cranial ultrasound video is calculated based on the confidence level of the anatomical structure category and the preset weight.

[0074] Specifically, the preset weights are pre-set to represent the typicality and clinical importance of anatomical structures in the corresponding standard sections. Each preset weight corresponds one-to-one with an anatomical structure category. After detecting the predicted anatomical structure category of a neonatal cranial ultrasound video frame, the preset weight corresponding to each neonatal cranial ultrasound video frame can be selected based on the correspondence between the preset weights and the anatomical structure categories.

[0075] After obtaining the preset weights, a predicted score can be calculated based on the confidence level of the anatomical structure category and the preset weights. The predicted score can be the result of the confidence level of the anatomical structure category and the preset weights, or a function relationship between the predicted score and the confidence level of the anatomical structure category and the preset weights can be pre-set. After obtaining the confidence level of the anatomical structure category and the preset weights, the predicted score can be calculated directly by calling this function relationship.

[0076] In step S32, each video frame cluster corresponds to a standard section, and each neonatal cranial ultrasound video frame in each video frame cluster is a candidate video frame for that standard section. That is, the anatomical structure prediction category of each neonatal cranial ultrasound video frame in each video frame cluster is the same, and the prediction score of each neonatal cranial ultrasound video frame meets the scoring requirements corresponding to the standard plane.

[0077] In one embodiment, dividing the neonatal cranial ultrasound video into several video frame clusters based on the predicted score and anatomical structure predicted category specifically includes:

[0078] Read the frame number of each neonatal cranial ultrasound video frame, and construct two-dimensional input data for each neonatal cranial ultrasound video frame based on the frame number and the predicted score;

[0079] Based on the anatomical structure prediction category, the neonatal cranial ultrasound video is clustered according to the two-dimensional input data to obtain several video frame clusters. Among them, the neonatal cranial ultrasound video frames in each video frame cluster have the same anatomical structure prediction category, are continuous in time, and have a prediction score higher than a preset score threshold.

[0080] Specifically, the frame number is the time index of the neonatal cranial ultrasound video frame within the neonatal cranial ultrasound video. This frame number determines the acquisition order between neonatal cranial ultrasound video frames. Since neonatal cranial ultrasound videos are typically acquired one standard section before moving to another, the acquisition order of neonatal cranial ultrasound video frames corresponding to the same standard section is generally continuous. Therefore, neonatal cranial ultrasound videos can be clustered into predicted categories for each anatomical structure based on the frame number and predicted score to obtain the video frame clusters corresponding to each standard section.

[0081] Based on this, this application embodiment constructs a two-dimensional input data including frame number and prediction score. Then, based on the anatomical structure prediction category, the neonatal cranial ultrasound video frames are grouped to obtain several video frame groups. Then, video frame filtering is performed on each video frame group using the two-dimensional input data as the filtering basis to obtain video frame clusters corresponding to each standard plane. Specifically, video frame filtering on each video frame group using the two-dimensional input data involves selecting video frames with consecutive frame numbers and prediction scores higher than a preset score threshold from the video frame group. That is, the frame numbers of each neonatal cranial ultrasound video frame in the video frame cluster are consecutive (i.e., continuous in time), and the prediction score of each neonatal cranial ultrasound video frame is higher than the preset score threshold. During video frame filtering, the DBSCAN clustering algorithm (e.g., eps = 10 frames, minimum sample size = 3, etc.) can be used to analyze the two-dimensional data to achieve video frame filtering.

[0082] In step S33, after obtaining the video frame cluster corresponding to each standard section, the neonatal cranial ultrasound video frame with the highest predicted score can be selected as the standard section image from the video frame cluster. Of course, when there are multiple neonatal cranial ultrasound video frames with the highest predicted scores, a neonatal cranial ultrasound video frame can be randomly selected as the standard section image, or the standard section image can be selected through interaction with the user, or the standard section image can be selected based on image clarity, etc.

[0083] This application embodiment uses anatomical structure prediction categories as a basis to cluster the neonatal cranial ultrasound video based on the two-dimensional input data. This can filter out abnormal neonatal cranial ultrasound video frames that are isolated outside the main cluster or appear at the wrong time and location, improving the accuracy of standard section images and thus improving the accuracy of brain injury prediction.

[0084] The above describes the process of identifying and extracting standard sections using the slice extraction model. The training process of the slice extraction model will be explained below.

[0085] For example, the training process of the slice extraction model specifically includes:

[0086] H10. Obtain cranial ultrasound video data from multiple neonatal cases;

[0087] H20. Select cranial ultrasound video data pairs from cranial ultrasound video data of multiple neonatal cases, and linearly mix the standard section images in each selected cranial ultrasound video data pair to obtain a mixed ultrasound image. The two cranial ultrasound video data pairs in the cranial ultrasound video data pair have the same number of standard sections.

[0088] H30. Based on the cranial ultrasound video data and all mixed ultrasound images of the multiple neonatal cases, the preset slice extraction model is trained to obtain the trained slice extraction model.

[0089] In step H10, since the standard section image can be calculated based on the anatomical structure prediction results, and the model parameters of the standard section extraction module do not need to be optimized during this calculation process, only the model parameters of the anatomical structure detection module need to be optimized when training the slice extraction model. Therefore, when constructing the training data, multiple cranial ultrasound video data sets, each containing several cranial ultrasound images, can be acquired, and anatomical structure annotation results can be configured for each cranial ultrasound image in each cranial ultrasound video data set. That is, the cranial ultrasound video data for each neonatal case includes several cranial ultrasound images, each cranial ultrasound image corresponds to the same neonatal case, and each cranial ultrasound image carries anatomical structure annotation results.

[0090] Furthermore, after acquiring cranial ultrasound video data from multiple neonatal cases, each cranial ultrasound image in the video data of each neonatal case can be standardized and data augmented. The standardization process is the same as described above and will not be repeated here. Data augmentation can be applied to the standardized cranial ultrasound images by random flipping (probability 0.5), rotation (angle range ±15 degrees), Gaussian blur (kernel size 5×5, standard deviation between 0.1 and 5, application probability 0.2), and grayscale perturbation (probability 0.2).

[0091] In step H20, after acquiring cranial ultrasound video data from multiple neonatal cases, a mixed ultrasound image is obtained through cross-case multi-view blending enhancement. The cross-case multi-view blending enhancement process involves first grouping the cranial ultrasound video data according to the number of standard sections included, resulting in several cranial ultrasound video data groups. Then, the cranial ultrasound video data within each group are randomly paired to obtain several cranial ultrasound video data pairs. Finally, the standard section images corresponding to the same standard section in the cranial ultrasound video data pairs are linearly blended to obtain the mixed ultrasound image. The calculation formula for the mixed ultrasound image is as follows:

[0092] mixed img =β·A img +(1-β)·B img ,

[0093] Among them, mixed img Represents a mixed ultrasound image, A img B represents a standard cross-sectional image. img Indicates another card with A img The standard cross-sectional image corresponding to the same standard cross-section, where α is randomly sampled from the Beta distribution (e.g., β = 0.4).

[0094] In step H30, after acquiring the mixed ultrasound image, a preset slice extraction model is trained based on cranial ultrasound video data from multiple neonatal cases and all mixed ultrasound images. The mixed ultrasound images do not contain anatomical annotations; they are used in conjunction with the two standard cross-sectional images used to generate the mixed ultrasound image during training. Specifically, when training the preset slice extraction model using cranial ultrasound video data from multiple neonatal cases and all mixed ultrasound images, for cranial ultrasound images not used to generate the mixed ultrasound image, these images are directly used as input to the preset slice extraction model. For cranial ultrasound images used to generate the mixed ultrasound image, both the cranial ultrasound image and the mixed ultrasound image are used as input to the preset slice extraction model.

[0095] Based on this, in one embodiment, training a preset slice extraction model based on the cranial ultrasound video data of the plurality of neonatal cases and the mixed ultrasound images to obtain a trained slice extraction model specifically includes:

[0096] For each cranial ultrasound image in each cranial ultrasound video data, a target mixed ultrasound image corresponding to the cranial ultrasound image is found among all mixed ultrasound images, wherein the target mixed ultrasound image is generated based on the cranial ultrasound image.

[0097] For the cranial ultrasound image where the target mixed ultrasound image is found, the preset slice extraction model is trained by combining the cranial ultrasound image and the target mixed ultrasound image;

[0098] For cranial ultrasound images for which no target mixed ultrasound image was found, a preset slice extraction model is trained based on the cranial ultrasound image.

[0099] Specifically, training the preset slice extraction model by combining the cranial ultrasound image and the target mixed ultrasound image involves simultaneously inputting the cranial ultrasound image and the target mixed ultrasound image into the preset slice extraction model. The model then outputs the anatomical structure prediction results corresponding to each of the cranial ultrasound image and the target mixed ultrasound image. A loss function is then constructed using the anatomical structure prediction results of the cranial ultrasound image and the target mixed ultrasound image, the anatomical structure annotation results corresponding to the cranial ultrasound image, and the anatomical structure annotation results corresponding to another cranial ultrasound image used to generate the target mixed ultrasound image. This loss function is then used to train the preset slice extraction model. The loss function for training the preset slice extraction model by combining the cranial ultrasound image and the target mixed ultrasound image can be expressed as:

[0100]

[0101] in, Let α represent the loss function, and p represent the weighting coefficients. A Represents a cranial ultrasound image, p mixed Represents a mixed ultrasound image, y A This represents the anatomical annotation results of a cranial ultrasound image, y B This represents the anatomical annotation results of another cranial ultrasound image, which is then mixed with the aforementioned cranial ultrasound image to generate the hybrid ultrasound image. Indicates based on y A ,y A ;p A ,p mixed The constructed loss function, Based on y A ,y B ;p A ,p mixed The constructed loss function.

[0102] further, To make y A With p A loss term and y A and p mixed The loss term is calculated according to a preset calculation method (such as a weighted calculation method). y A With p A loss term and y B and p mixedThe loss term is calculated according to a preset calculation method (such as a weighted calculation method). The loss term can include a Varifocal loss term, an L1 loss term, and a DIoU loss term. The Varifocal loss term is the classification cost, used to measure the difference between the predicted class confidence and the true label for learning classification confidence. The L1 loss term is the bounding box regression cost, used to measure the difference between the predicted box and the true box in terms of center point coordinates and size. The DIoU loss term is the orientation alignment cost, used to accurately evaluate the difference between the two rotated boxes in terms of spatial overlap and center point distance.

[0103] Furthermore, to improve the training stability of deep networks, an auxiliary supervisor is connected after each decoding layer of the Transformer decoder in the decoding unit during training. This auxiliary supervisor is used to predict brain lesions for that decoding layer to obtain the corresponding brain lesion prediction result. Then, a comprehensive cost matrix is ​​constructed based on the brain lesion prediction results and brain lesion annotation results of each decoding layer. The comprehensive cost matrix includes the Varifocal loss term, L1 loss term, and DIoU loss term determined for each decoding layer. This comprehensive cost matrix is ​​then input into the Hungarian algorithm to find the optimal allocation scheme that minimizes the total matching cost. This allows for precise alignment of the brain lesion annotation box with a brain lesion prediction box, thereby providing accurate supervision signals for subsequent loss calculations.

[0104] In one embodiment, a contrastive denoising training strategy is introduced during the training phase. This strategy introduces a noisy query to enable the model to learn how to recover brain injury annotation results from the noisy query, thereby improving the model's noise resistance and performance. Specifically, the brain injury annotation results are used as a known query, and controllable noise is applied to this known query to generate a noisy query. The model is then guided to learn how to recover the brain injury annotation results from the noisy query. This involves constructing a denoising loss term using the denoised query generated by the model based on the noisy query and the known query, and then using this denoising loss term to optimize the model. The controllable noise includes brain injury annotation box coordinate and size noise, and rotation angle noise. The brain injury annotation box coordinate and size noise is achieved by adding independent uniform noise to the center coordinates, width, and height of the rotated box within the brain injury annotation box. The noise range is [-scale×d, scale×d], where d is the normalization factor for the image diagonal length, and scale is an adjustable hyperparameter. Rotation angle noise adds uniform noise within the range of [-Δθ, Δθ] to the rotation angle of the brain injury annotation box. Positive sample paths use smaller perturbations (e.g., Δθ = 20°) and negative sample paths use larger perturbations (e.g., Δθ = 40°).

[0105] S40. Based on all extracted standard cross-sectional images, the brain injury prediction model outputs a predicted brain injury category.

[0106] Specifically, all standard section images can include three standard coronal sections and three standard sagittal sections. In practice, however, some of these standard sections may be included. The brain injury prediction model is used to predict brain injury based on these standard section images. The predicted brain injury categories include image-level and case-level categories. Image-level categories include normal, ependymal cyst, ventricular dilatation, hydrocephalus, leukomalacia, and intraventricular hemorrhage. Case-level categories include severe and mild brain injury. Severe brain injury includes grade III-IV intraventricular hemorrhage, hydrocephalus, and diffuse leukomalacia, which require clinical intervention.

[0107] For example, the step of outputting a predicted brain injury category through the brain injury prediction model based on all extracted standard cross-sectional images specifically includes:

[0108] All extracted standard cross-sectional images are input into the brain injury prediction model, and the brain injury prediction model determines the image-level predicted brain injury category corresponding to each standard cross-sectional image.

[0109] Based on the image-level prediction of brain injury categories corresponding to all standard cross-sectional images, the case-level predicted brain injury category is determined by the brain injury prediction model.

[0110] Specifically, the brain injury prediction model includes a feature extraction module, an image multi-label classification module, and a case-level classification module. The feature extraction module extracts features from standard cross-sectional images to obtain high-dimensional features. The image multi-label classification module uses these high-dimensional features to perform image-level brain injury prediction for each frame of neonatal craniotomy video. The case-level classification module performs case-level brain injury prediction based on neonatal craniotomy video. The feature extraction module has the same network structure as the feature extraction unit in the anatomical structure detection module.

[0111] The image multi-label classification module consists of a multi-label classification prediction head and an activation function layer. The multi-label classification prediction head is used to predict brain injury based on the high-dimensional features of a standard cross-sectional image. The activation function layer uses the sigmoid activation function to determine the prediction confidence and outputs a multi-dimensional vector representing the probability of the standard cross-sectional image in each brain injury category. During the training of the brain injury prediction model, the Binary Cross Entropy loss function is used for multi-label supervision of the image multi-label classification module.

[0112] The case-level classification module integrates predicted brain injury categories from all standard cross-sectional images to form a case-level predicted brain injury category. Specifically, a learnable classification token is first created for the neonatal case. Then, this classification token, along with the high-dimensional features of all standard cross-sectional images, forms a 7-token input sequence, which is fed into the multi-section feature fusion unit, composed of a Transformer encoder, in the case-level classification module. This multi-section feature fusion unit models the spatial and semantic relationships between different cross-sectional features through a self-attention mechanism, achieving information interaction and aggregation. Finally, the output corresponding to the classification token represents the case-level features that fuse all cross-sectional information. These case-level features are then fed into the classification head to predict the neonatal case-level predicted brain injury category.

[0113] This application improves the accuracy of brain injury prediction by obtaining the image-level predicted brain injury category corresponding to each standard cross-sectional image and the case-level predicted brain injury category determined based on all standard cross-sectional images, and then combining the image-level predicted brain injury category and the case-level predicted brain injury category for comprehensive judgment.

[0114] In one embodiment, to improve prediction accuracy, data augmentation can be applied to the standard cross-sectional images when the brain injury prediction model outputs a predicted brain injury category based on all extracted standard cross-sectional images. Specifically, for each standard cross-sectional image, a preset number (e.g., 10) of perturbed standard cross-sectional images can be generated. Then, the standard cross-sectional image and all perturbed standard cross-sectional images are input into the brain injury prediction model. The brain injury prediction model outputs the prediction confidence scores of the input standard cross-sectional image and each perturbed standard cross-sectional image. The mean of all prediction confidence scores is then calculated to obtain the fusion prediction probability, and the image-level predicted brain injury category corresponding to the standard cross-sectional image is determined based on this fusion prediction probability.

[0115] In one embodiment, when outputting a predicted brain injury category through the brain injury prediction model, a key region heatmap can also be determined. Simultaneously, a standard cross-sectional image can be correlated with the key region heatmap to generate a predicted image containing the standard cross-section and the corresponding visualized region of interest. The key region heatmap is generated using the Finer-CAM method, which calculates the gradient of the high-dimensional feature map at the input of the multi-section feature fusion module (i.e., the Transformer encoder) in the diagnostic module relative to the final predicted score for the "severe brain injury" category, thereby locating the key anatomical regions of interest to the model during decision-making.

[0116] In summary, this embodiment provides a brain injury prediction method based on neonatal cranial ultrasound images. This method is the first to propose a multi-view joint prediction strategy for neonatal cranial ultrasound, fusing multiple standard cross-sectional images from coronal and sagittal planes to achieve comprehensive case-level judgment. It exhibits high predictive accuracy and significant clinical applicability. Furthermore, this embodiment combines standard cross-sectional image extraction with brain injury category prediction, utilizing multiple standard cross-sectional images for brain injury category prediction. This provides richer semantic information for brain injury category prediction, significantly improving accuracy and consistency in identifying severe brain injuries. It also significantly reduces the average screening time per case compared to manual interpretation, significantly improving work efficiency. In addition, this method has the capability to automatically process low-quality or blind scan ultrasound videos and extract standard cross-sectional images, making it suitable for initial ultrasound screening tasks in grassroots or resource-scarce areas.

[0117] Based on the aforementioned brain injury prediction method using neonatal cranial ultrasound images, this embodiment provides a brain injury prediction device based on neonatal cranial ultrasound images. The device utilizes a trained brain injury prediction model, which includes a slice extraction model and a brain injury prediction model. The slice extraction model includes an anatomical structure detection module and a standard section extraction module, such as... Figure 3 As shown, the brain injury prediction device based on neonatal cranial ultrasound images specifically includes:

[0118] Acquisition module 100 is used to acquire ultrasound videos of the newborn's cranium;

[0119] The first control module 200 is used to determine the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video through the anatomical structure detection module based on the neonatal cranial ultrasound video; and to extract standard section images from the neonatal cranial ultrasound video through the standard section extraction module based on the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video.

[0120] The second control module 300 is used to output a predicted brain injury category based on all extracted standard cross-sectional images through the brain injury prediction model.

[0121] Based on the above-described method for predicting brain injury based on neonatal cranial ultrasound images, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the method for predicting brain injury based on neonatal cranial ultrasound images as described in the above embodiment.

[0122] Based on the above-mentioned brain injury prediction method based on neonatal cranial ultrasound images, this application also provides a terminal device, such as... Figure 4As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0123] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0124] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0125] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0126] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for predicting brain injury based on neonatal cranial ultrasound images, characterized in that, The brain injury prediction model, which includes a slice extraction model and a brain injury prediction model, is applied using a trained brain injury prediction model. The slice extraction model includes an anatomical structure detection module and a standard section extraction module. Specifically, the brain injury prediction method based on neonatal cranial ultrasound images includes: Obtaining ultrasound video of the newborn's brain; Based on the neonatal cranial ultrasound video, the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video is determined by the anatomical structure detection module. Based on the anatomical structure prediction results of each frame of neonatal cranial ultrasound video, standard section images are extracted from the neonatal cranial ultrasound video using the standard section extraction module. Based on all extracted standard cross-sectional images, the brain injury prediction model outputs a predicted brain injury category.

2. The method for predicting brain injury based on neonatal cranial ultrasound images according to claim 1, characterized in that, The step of extracting standard section images from the neonatal cranial ultrasound video based on the anatomical structure prediction results of each frame of neonatal cranial ultrasound video, using the standard section extraction module, specifically includes: The predicted score for each neonatal cranial ultrasound video frame is determined based on the confidence level of the anatomical structure category of each frame. The neonatal cranial ultrasound video is divided into several video frame clusters based on the predicted score and anatomical structure prediction category, wherein each video frame cluster corresponds to a standard section. In each video frame cluster, a video frame is selected according to the predicted score, and this video frame is used as the standard cross-sectional image corresponding to the video frame cluster.

3. The method for predicting brain injury based on neonatal cranial ultrasound images according to claim 2, characterized in that, The anatomical structure prediction results include anatomical structure category confidence scores; the determination of the prediction score for each frame of neonatal cranial ultrasound video based on the anatomical structure category confidence scores specifically includes: The preset weights corresponding to each frame of neonatal cranial ultrasound video are determined based on the predicted anatomical structure category. The predicted score for each frame of neonatal cranial ultrasound video is calculated based on the confidence level of the anatomical structure category and the preset weight.

4. The method for predicting brain injury based on neonatal cranial ultrasound images according to claim 2 or 3, characterized in that, The step of dividing the neonatal cranial ultrasound video into several video frame clusters based on the predicted score and anatomical structure predicted category specifically includes: Read the frame number of each neonatal cranial ultrasound video frame, and construct two-dimensional input data for each neonatal cranial ultrasound video frame based on the frame number and the predicted score; Based on the anatomical structure prediction category, the neonatal cranial ultrasound video is clustered according to the two-dimensional input data to obtain several video frame clusters. Among them, the neonatal cranial ultrasound video frames in each video frame cluster have the same anatomical structure prediction category, are continuous in time, and have a prediction score higher than a preset score threshold.

5. The method for predicting brain injury based on neonatal cranial ultrasound images according to claim 1, characterized in that, The training process of the slice extraction model specifically includes: We acquired cranial ultrasound video data from multiple neonatal cases. Each cranial ultrasound video data set included several cranial ultrasound images, and each cranial ultrasound image carried anatomical structure annotations. Cranial ultrasound video data pairs were selected from multiple neonatal cases, where the two cranial ultrasound video data pairs had the same number of standard sections. The standard section images in each selected pair of cranial ultrasound video data are linearly blended to obtain a mixed ultrasound image; Based on the cranial ultrasound video data and all mixed ultrasound images of the multiple neonatal cases, a preset slice extraction model was trained to obtain a trained slice extraction model.

6. The method for predicting brain injury based on neonatal cranial ultrasound images according to claim 5, characterized in that, The step of training a preset slice extraction model based on the cranial ultrasound video data of the multiple neonatal cases and the mixed ultrasound images to obtain a trained slice extraction model specifically includes: For each cranial ultrasound image in each cranial ultrasound video data, a target mixed ultrasound image corresponding to the cranial ultrasound image is found among all mixed ultrasound images, wherein the target mixed ultrasound image is generated based on the cranial ultrasound image. For the cranial ultrasound image where the target mixed ultrasound image is found, the preset slice extraction model is trained by combining the cranial ultrasound image and the target mixed ultrasound image; For cranial ultrasound images where no target mixed ultrasound image was found, a preset slice extraction model was trained based on the cranial ultrasound image. The loss function for training the preset slice extraction model using the combined cranial ultrasound image and its corresponding hybrid ultrasound image is: in, Let α represent the loss function, and p represent the weighting coefficients. A Represents a cranial ultrasound image, p mixed Represents a mixed ultrasound image, y A This represents the anatomical annotation results of a cranial ultrasound image, y B This represents the anatomical annotation results of another cranial ultrasound image, which is then mixed with the aforementioned cranial ultrasound image to generate the hybrid ultrasound image. Indicates based on y A ,y A ;p A ,p mixed The constructed loss function, Based on y A ,y B ;p A ,p mixed The constructed loss function.

7. The method for predicting brain injury based on neonatal cranial ultrasound images according to claim 1, characterized in that, The step of outputting a predicted brain injury category based on all extracted standard cross-sectional images through the brain injury prediction model specifically includes: All extracted standard cross-sectional images are input into the brain injury prediction model, and the brain injury prediction model determines the image-level predicted brain injury category corresponding to each standard cross-sectional image. Based on the image-level prediction of brain injury categories corresponding to all standard cross-sectional images, the case-level predicted brain injury category is determined by the brain injury prediction model.

8. A brain injury prediction device based on neonatal cranial ultrasound images, characterized in that, The brain injury prediction model, which includes a slice extraction model and a brain injury prediction model, is applied. The slice extraction model includes an anatomical structure detection module and a standard section extraction module. Specifically, the brain injury prediction device based on neonatal cranial ultrasound images includes: The acquisition module is used to acquire ultrasound videos of the newborn's cranium; The first control module is used to determine the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video through the anatomical structure detection module based on the neonatal cranial ultrasound video; and to extract standard section images from the neonatal cranial ultrasound video through the standard section extraction module based on the anatomical structure prediction result of each frame of the neonatal cranial ultrasound video. The second control module is used to output a predicted brain injury category based on all extracted standard cross-sectional images through the brain injury prediction model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the brain injury prediction method based on neonatal cranial ultrasound images as described in any one of claims 1-7.

10. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the brain injury prediction method based on neonatal cranial ultrasound images as described in any one of claims 1-7.

Images