A three-dimensional ultrasonic volume data processing method, device and equipment and medium

Abstract

This application discloses a method, apparatus, electronic device, and computer-readable storage medium for processing three-dimensional ultrasound volume data. The method includes: acquiring three-dimensional ultrasound volume data; determining a two-dimensional ultrasound image from the three-dimensional ultrasound volume data; inputting the two-dimensional ultrasound image into an examination object recognition network to determine a target examination object corresponding to the two-dimensional ultrasound image using the examination object recognition network; and inputting the three-dimensional ultrasound volume data into a three-dimensional examination network corresponding to the target examination object to determine the examination result using the three-dimensional examination network. This application improves the efficiency of ultrasound examination.

Description

Technical Field

[0001] This application relates to the field of ultrasound examination technology, and more specifically, to a three-dimensional ultrasound body data processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Currently, ultrasound equipment has integrated multiple functions, such as 3D fetal face display, 3D follicle counting, 3D endometrial segmentation, and 3D pelvic floor measurement. Each function has its own unique application scenario. Doctors need to constantly switch application scenarios when using different functions, resulting in low examination efficiency and high operational complexity.

[0003] Therefore, how to improve the efficiency of ultrasound examination is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a three-dimensional ultrasound body data processing method, apparatus, electronic device, and computer-readable storage medium, which improves the efficiency of ultrasound examination.

[0005] To achieve the above objectives, this application provides a three-dimensional ultrasound body data processing method, comprising:

[0006] Acquire three-dimensional ultrasound volume data, and determine two-dimensional ultrasound images from the three-dimensional ultrasound volume data;

[0007] The two-dimensional ultrasound image is input into the examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image.

[0008] The three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to determine the examination result using the three-dimensional examination network.

[0009] The process of inputting the two-dimensional ultrasound image into an examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image includes:

[0010] The two-dimensional ultrasound image is input into the examination object recognition network;

[0011] In the object recognition network, the first semantic features of the two-dimensional ultrasound image are extracted based on the first feature extraction branch; wherein, the first feature extraction branch includes one or more first feature extraction modules for extracting semantic features of the channel dimension;

[0012] In the object recognition network, the second semantic features of the two-dimensional ultrasound image are extracted based on the second feature extraction branch; wherein, the second feature extraction branch includes one or more second feature extraction modules for extracting semantic features of the spatial dimension;

[0013] The target semantic features of the two-dimensional ultrasound image are obtained by fusing the first semantic features and the second semantic features through a pooling layer;

[0014] The target examination object of the two-dimensional ultrasound image is determined based on the target semantic features.

[0015] Wherein, if the first feature extraction branch includes N first feature extraction modules and the second feature extraction branch includes N second feature extraction modules, then the input feature of the i-th second feature extraction module in the second feature extraction branch is a fusion feature of the input feature of the (i-1)-th second feature extraction module, the output feature of the (i-1)-th second feature extraction module, and the output feature of the (i-1)-th first feature extraction module. The input feature of the pooling layer includes the output feature of the N-th first feature extraction module, the output feature of the N-th second feature extraction module, and the input feature of the N-th second feature extraction module; 2≤i≤N, N≥2.

[0016] The process of inputting the three-dimensional ultrasound data into a three-dimensional examination network corresponding to the target object, and using the three-dimensional examination network to determine the examination results, includes:

[0017] The three-dimensional ultrasound body data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location area of ​​the examination site corresponding to the target examination object;

[0018] The inspection result corresponding to the target inspection object is determined based on the location area of ​​the inspection site.

[0019] Wherein, if the target examination object is the pelvic floor, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location region of the examination site corresponding to the target examination object, including:

[0020] The three-dimensional ultrasound data is input into a three-dimensional pelvic floor examination network to determine the urethral region using the three-dimensional pelvic floor examination network.

[0021] Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes:

[0022] Determine multiple two-dimensional slices of the urethral region, and determine the center point of each of the multiple two-dimensional slices;

[0023] The center points of the multiple two-dimensional slices are connected according to their arrangement order in the urethral region to show the location of the urethra.

[0024] Where the target examination object is a fetus, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location region of the examination site corresponding to the target examination object, including:

[0025] The three-dimensional ultrasound data is input into a three-dimensional fetal examination network to determine the fetal head region using the three-dimensional fetal examination network;

[0026] Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes:

[0027] Determine the key plane of the fetal head region, rotate the fetal head region so that the key plane is parallel to the screen to obtain the corrected fetal head region, and display the corrected fetal head region.

[0028] Wherein, if the target object for examination is a follicle, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target object, so as to use the three-dimensional examination network to determine the location region of the examination site corresponding to the target object, including:

[0029] The three-dimensional ultrasound data is input into a three-dimensional follicle detection network to determine multiple follicle regions using the three-dimensional follicle detection network;

[0030] Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes:

[0031] The volumes of multiple follicular regions are counted, and the multiple follicles are sorted from smallest to largest volume. The volumes and sorting results of the multiple follicular regions are then displayed.

[0032] Wherein, if the target examination object is the uterus, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location region of the examination site corresponding to the target examination object, including:

[0033] The three-dimensional ultrasound data is input into a three-dimensional uterine examination network to determine the endometrial region using the three-dimensional uterine examination network;

[0034] Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes:

[0035] An initial cutting template is determined based on the endometrial region; wherein, the voxel value of the voxel point belonging to the endometrial region in the initial cutting template is 1, and the voxel value of other voxel points is 0;

[0036] Determine the key surfaces in the initial cutting template, rotate the initial cutting template until the key surfaces are parallel to the screen to obtain the aligned cutting template, and record the rotation parameters during the rotation process;

[0037] Based on the rotation parameters, the three-dimensional ultrasound body data is rotated to obtain rotated three-dimensional ultrasound body data;

[0038] The voxel values ​​of the voxel points in the rotated three-dimensional ultrasound data are multiplied with the voxel values ​​of the corresponding voxel points in the aligned cutting template to obtain the aligned endometrial region, and the aligned endometrial region is displayed.

[0039] To achieve the above objectives, this application provides a three-dimensional ultrasound body data processing device, comprising:

[0040] The first determining module is used to acquire three-dimensional ultrasound volume data and determine a two-dimensional ultrasound image from the three-dimensional ultrasound volume data.

[0041] The second determining module is used to input the two-dimensional ultrasound image into the examination object recognition network, so as to use the examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image;

[0042] The third determining module is used to input the three-dimensional ultrasound body data into the three-dimensional examination network corresponding to the target examination object, so as to determine the examination result using the three-dimensional examination network.

[0043] To achieve the above objectives, this application provides an electronic device, comprising:

[0044] Memory, used to store computer programs;

[0045] A processor is used to execute the computer program to implement the steps of the three-dimensional ultrasound body data processing method described above.

[0046] To achieve the above objectives, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the three-dimensional ultrasound body data processing method described above.

[0047] As can be seen from the above scheme, the three-dimensional ultrasound body data processing method provided in this application includes: acquiring three-dimensional ultrasound body data, determining a two-dimensional ultrasound image from the three-dimensional ultrasound body data; inputting the two-dimensional ultrasound image into an examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image using the examination object recognition network; and inputting the three-dimensional ultrasound body data into a three-dimensional examination network corresponding to the target examination object to determine the examination result using the three-dimensional examination network.

[0048] The three-dimensional ultrasound body data processing method provided in this application automatically identifies the target examination object through an examination object recognition network, and automatically analyzes the three-dimensional ultrasound body data through the three-dimensional examination network corresponding to the target examination object to determine the examination result. Therefore, this application optimizes the examination process through automation, reducing the cumbersome operation of constantly switching application scenarios to activate different diagnostic functions, and improving the efficiency of ultrasound examination. This application also discloses a three-dimensional ultrasound body data processing device, an electronic device, and a computer-readable storage medium, which can achieve the same technical effects.

[0049] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings:

[0051] Figure 1 This is a flowchart illustrating a three-dimensional ultrasound body data processing method according to an exemplary embodiment;

[0052] Figure 2 This is a structural diagram of an object recognition network according to an exemplary embodiment;

[0053] Figure 3 This is a flowchart illustrating an ultrasound examination according to an exemplary embodiment;

[0054] Figure 4 This is a structural diagram of a three-dimensional ultrasound body data processing device according to an exemplary embodiment;

[0055] Figure 5This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, in the embodiments of this application, "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0057] This application discloses a three-dimensional ultrasound body data processing method, which improves the efficiency of ultrasound examination.

[0058] See Figure 1 A flowchart illustrating a three-dimensional ultrasound body data processing method according to an exemplary embodiment is shown below. Figure 1 As shown, it includes:

[0059] S101: Acquire three-dimensional ultrasound volume data, and determine a two-dimensional ultrasound image from the three-dimensional ultrasound volume data;

[0060] The execution entity in this embodiment can be an ultrasound device or a computing device connected to the ultrasound device. In this embodiment, the three-dimensional ultrasound volume data is the three-dimensional data of the target object acquired by the ultrasound device based on three-dimensional ultrasound imaging technology. The target object can be the uterus, follicles, fetus, pelvic floor, etc. Specifically, the ultrasound device emits ultrasound waves towards the area to be detected using an ultrasound probe. The ultrasound device reconstructs the three-dimensional volume data based on the reflected ultrasound echo data to obtain the three-dimensional ultrasound volume data. The three-dimensional ultrasound volume data can be generated in real time or based on historical ultrasound echo data.

[0061] In practice, a two-dimensional ultrasound image is derived from three-dimensional ultrasound volume data. As a feasible implementation method, the sagittal, transverse, or coronal planes of the three-dimensional ultrasound volume data can be used to determine the two-dimensional ultrasound image.

[0062] S102: Input the two-dimensional ultrasound image into the examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image using the examination object recognition network;

[0063] In this step, the two-dimensional ultrasound image is input into the examination object recognition network. The examination object recognition network is used to analyze the two-dimensional ultrasound image and identify the target examination object in the two-dimensional ultrasound image, thereby realizing the automatic detection and recognition of the examination object.

[0064] As a feasible implementation, the two-dimensional ultrasound image is input into an examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image. This includes: inputting the two-dimensional ultrasound image into the examination object recognition network; extracting first semantic features of the two-dimensional ultrasound image based on a first feature extraction branch in the examination object recognition network; wherein the first feature extraction branch includes one or more first feature extraction modules for extracting semantic features in the channel dimension; extracting second semantic features of the two-dimensional ultrasound image based on a second feature extraction branch in the examination object recognition network; wherein the second feature extraction branch includes one or more second feature extraction modules for extracting semantic features in the spatial dimension; fusing the first semantic features and the second semantic features through a pooling layer to obtain the target semantic features of the two-dimensional ultrasound image; and determining the target examination object of the two-dimensional ultrasound image based on the target semantic features.

[0065] In practical implementation, a two-dimensional ultrasound image is input into an examination object recognition network. This network extracts features from the image through two independent branches: a first feature extraction branch extracts semantic features along the channel dimension, while a second feature extraction branch focuses on extracting semantic features along the spatial dimension. The first feature extraction branch includes one or more first feature extraction modules responsible for extracting semantic features along the channel dimension. These modules can be SE Blocks (Squeeze-and-Excitation Blocks), which adaptively adjust the weights between channels to improve feature representation capabilities. The second feature extraction branch includes one or more second feature extraction modules responsible for extracting semantic features along the spatial dimension. These modules can be Transformer Blocks, which capture long-distance dependencies through a self-attention mechanism. The SE Block and Transformer Block improve image classification accuracy. The features extracted by these two branches are fused into target semantic features, which more comprehensively describe the target examination object in the two-dimensional ultrasound image. The examination object recognition network identifies the target examination object based on these semantic features, such as the uterus, follicles, fetus, and pelvic floor.

[0066] In a preferred embodiment, if the first feature extraction branch includes N first feature extraction modules and the second feature extraction branch includes N second feature extraction modules, then the input feature of the i-th second feature extraction module in the second feature extraction branch is a fusion feature of the input feature of the (i-1)-th second feature extraction module, the output feature of the (i-1)-th second feature extraction module, and the output feature of the (i-1)-th first feature extraction module. The input feature of the pooling layer includes the output feature of the N-th first feature extraction module, the output feature of the N-th second feature extraction module, and the input feature of the N-th second feature extraction module; 2≤i≤N, N≥2.

[0067] In practical implementation, if the first feature extraction branch includes N first feature extraction modules and the second feature extraction branch includes N second feature extraction modules, then for the i-th second feature extraction module, its input feature is a fusion of the input feature of the (i-1)-th second feature extraction module, the output feature of the (i-1)-th second feature extraction module, and the output feature of the (i-1)-th first feature extraction module. For example, for the second second feature extraction module, its input feature is a fusion of the input feature of the first second feature extraction module, the output feature of the first second feature extraction module, and the output feature of the first first feature extraction module. Furthermore, the fused output feature of the N-th second feature extraction module, the output feature of the N-th second feature extraction module, and the input feature of the N-th second feature extraction module can be processed by a pooling layer, such as GAP (Global Average Pooling). Then, the target inspection object is determined based on the processed features. GAP can reduce the number of parameters, reduce computation, and reduce overfitting.

[0068] By fusing the input features of the (i-1)th second feature extraction module, the output features of the (i-1)th second feature extraction module, and the output features of the (i-1)th first feature extraction module, the i-th second feature extraction module can extract more complex features. This allows the object recognition network to learn more complex feature interactions, improving its understanding of image content. Since the second feature extraction module can receive features from different sources, the object recognition network has greater flexibility and expressive power in feature extraction. By fusing features from different levels, the object recognition network can better handle noise and variations in images, thus improving its robustness. Simultaneously, it can increase the link length of the first and second feature extraction branches, meaning the first feature extraction branch contains more first feature extraction modules and the second feature extraction branch contains more second feature extraction modules, ensuring that each first and second feature extraction module can extract effective feature information. Because the model can capture richer features, this helps improve the accuracy of model recognition.

[0069] A structural diagram of an object recognition network is shown below. Figure 2 As shown, the input feature of the i-th SE Block is the output feature of the (i-1)-th SE Block, and the input feature of the i-th Transformer Block is the fused feature of the input feature of the (i-1)-th Transformer Block, the output feature of the (i-1)-th Transformer Block, and the output feature of the (i-1)-th SE Block. The output feature of the N-th SE Block, the output feature of the N-th Transformer Block, and the input feature of the N-th Transformer Block are fused and then processed by GAP.

[0070] The training process of the object recognition network is as follows: First, the object recognition network is constructed, including a first feature extraction branch, a second feature extraction branch, and a pooling layer. The first feature extraction branch includes one or more first feature extraction modules, which are responsible for extracting semantic features in the channel dimension. These first feature extraction modules can be SEBlocks (Squeeze-and-Excitation Blocks), which improve feature representation by adaptively adjusting the weights between channels. The second feature extraction branch includes one or more second feature extraction modules, which are responsible for extracting semantic features in the spatial dimension. These second feature extraction modules can be TransformerBlocks, which capture long-distance dependencies through a self-attention mechanism. The pooling layer is responsible for fusing the features extracted by the two branches into the target semantic features. Next, a suitable image dataset for training is collected. To improve the model's generalization ability, data augmentation techniques such as rotation, scaling, and cropping are used to increase data diversity and reduce overfitting. The training dataset is preprocessed by converting the original image data into a format that the model can process and normalizing the image data to a certain range, typically 0 to 1 or -1 to 1, to facilitate network processing. Normalization helps accelerate the training process and improve the model's convergence speed. Image sizes are adjusted according to the requirements of the network's input layers to ensure all input images have the same dimensions. Then, the training environment is configured, hyperparameters are set, the loss function is defined, and a suitable optimizer is selected. Appropriate batch size and iteration counts are set to ensure the network can learn effectively. The training image data is labeled with the objects to be inspected. This labeled data is then input into the object recognition network, which predicts the objects to be inspected in each training image. The network is trained based on the labeled and predicted objects in each training image. Finally, the network is evaluated using metrics such as precision, recall, and intersection over union (IOU) to assess its recognition performance and ensure accurate object identification.

[0071] S103: Input the three-dimensional ultrasound body data into the three-dimensional examination network corresponding to the target examination object, so as to determine the examination result using the three-dimensional examination network.

[0072] In practice, different examination objects correspond to different three-dimensional examination networks. Since the target examination object corresponding to the three-dimensional ultrasound body data was determined in the previous step, the three-dimensional examination network corresponding to the target examination object was also determined. The original three-dimensional ultrasound body data was input into the three-dimensional examination network corresponding to the target examination object. This three-dimensional examination network can automatically analyze the three-dimensional ultrasound body data and determine the examination results for the target examination object.

[0073] As a feasible implementation method, inputting the three-dimensional ultrasound data into a three-dimensional examination network corresponding to the target examination object to determine the examination result using the three-dimensional examination network includes: inputting the three-dimensional ultrasound data into a three-dimensional examination network corresponding to the target examination object to determine the location region of the examination site corresponding to the target examination object using the three-dimensional examination network; and determining the examination result corresponding to the target examination object based on the location region of the examination site.

[0074] In practice, the raw 3D ultrasound data is input into the 3D examination network corresponding to the target object. This network automatically analyzes the input ultrasound data and determines the location of the examination area corresponding to the target object. The 3D examination network is a deep learning model that understands and analyzes the complex structures and features in the 3D ultrasound data, thereby accurately locating specific areas to be examined, such as the urethra, fetal head, follicles, and endometrium. Once the location of these areas is determined, the network further analyzes their characteristics, such as shape, size, and density, to determine the examination results for the target object.

[0075] The ultrasound examination procedure in this embodiment is as follows: Figure 3 As shown, the process begins by identifying a two-dimensional ultrasound image from the three-dimensional ultrasound data. This two-dimensional ultrasound image is then processed by a 2D recognition network to determine if the target object is the uterus. If it is the uterus, the endometrial segmentation network determines the examination results. If it is not the uterus, the process moves to determine if the target object is a follicle. If it is a follicle, the follicle counting network determines the examination results. If it is not a follicle, the process moves to determine if the target object is a fetus. If it is a fetus, the 3D fetal display network determines the examination results. If it is not a fetus, the process moves to determine if the target object is the pelvic floor. If it is the pelvic floor, the 3D pelvic floor measurement network determines the examination results. If it is not the pelvic floor, the process ends.

[0076] The three-dimensional ultrasound body data processing method provided in this application automatically identifies the target examination object through an examination object recognition network, and automatically analyzes the three-dimensional ultrasound body data through the three-dimensional examination network corresponding to the target examination object to determine the examination result. Therefore, this application embodiment optimizes the examination process through automation, reduces the cumbersome operation of constantly switching application scenarios to activate different diagnostic functions, and improves the efficiency of ultrasound examination.

[0077] Based on the above embodiments, as a feasible implementation method, if the target examination object is the pelvic floor, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object to determine the location region of the examination site corresponding to the target examination object using the three-dimensional examination network. This includes: inputting the three-dimensional ultrasound data into the three-dimensional pelvic floor examination network to determine the urethral region using the three-dimensional pelvic floor examination network; correspondingly, determining the examination result corresponding to the target examination object based on the location region of the examination site includes: determining multiple two-dimensional slices of the urethral region and determining the center points of the multiple two-dimensional slices; connecting the center points of the multiple two-dimensional slices according to their arrangement order in the urethral region to display the location of the urethra.

[0078] In practical implementation, if the target object identified by the examination object recognition network is the pelvic floor, then the corresponding examination site is the urethra. That is, the 3D pelvic floor examination network is used to determine and extract the urethral region from the 3D ultrasound data. For the urethral region, the 3D pelvic floor examination network determines a cross-sectional direction to ensure the urethra is displayed most clearly in the 2D slice. For each 2D cross-section, traditional image processing algorithms are used to determine the center point on the cross-section. The center points calculated on all 2D cross-sections are then connected according to their order in 3D space. This yields a continuous path representing the position and direction of the urethra in 3D space. In other words, if the target object identified by the examination object recognition network is the pelvic floor, then the corresponding examination result will show the location of the urethra.

[0079] Based on the above embodiments, as a feasible implementation method, if the target examination object is a fetus, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object to determine the location region of the examination site corresponding to the target examination object using the three-dimensional examination network. This includes: inputting the three-dimensional ultrasound data into the three-dimensional fetal examination network to determine the fetal head region using the three-dimensional fetal examination network; correspondingly, determining the examination result corresponding to the target examination object based on the location region of the examination site includes: determining the key plane of the fetal head region, rotating the fetal head region to a direction where the key plane is parallel to the screen to obtain the aligned fetal head region, and displaying the aligned fetal head region.

[0080] In practical implementation, if the target object identified by the examination object recognition network is a fetus, then the corresponding examination area is the fetal head region. That is, the 3D fetal examination network is used to determine voxel points belonging to the fetal head region in the 3D ultrasound data. Based on these voxel points, a target to be rotated is constructed. Key surfaces, i.e., the optimal viewing surfaces, are determined based on key information on the target. This key information can include key point locations and structural information. Then, through calculation and application of rotation transformation, the target is rotated to the target orientation to obtain the rotated target. Based on the rotated target, the aligned fetal head region is determined. The voxel values ​​of each voxel point in the aligned fetal head region are consistent with the voxel values ​​of the voxel points in the 3D ultrasound data acquired by the ultrasound equipment. The aligned fetal head region can then be displayed using the ultrasound equipment.

[0081] As a feasible implementation method, the segmented region corresponding to the fetal head region can be directly extracted from the original three-dimensional ultrasound data based on the voxel points belonging to the fetal head region as the target to be rotated. At this time, the voxel values ​​of each voxel point in the target to be rotated are consistent with the voxel values ​​of the voxel points in the original three-dimensional ultrasound data. After rotating the target to be rotated to the target direction, the rotated target is the corrected fetal head region.

[0082] As another feasible implementation, an initial clipping template is first created. In this initial clipping template, the voxel values ​​of voxel points belonging to the fetal head region are 1, while the voxel values ​​of other voxel points are 0. Then, the initial clipping template is designated as the target to be rotated. The target to be rotated is rotated to the target direction to obtain the rotated target, and the rotation parameters during the rotation process are recorded. At this point, the voxel values ​​of each voxel point in the rotated target are either 1 or 0, which do not correspond to the voxel values ​​of the voxel points in the original 3D ultrasound volume data. Therefore, in order to obtain the aligned fetal head region in the 3D ultrasound volume data, the same rotation operation needs to be performed on the original 3D ultrasound volume data using the aforementioned recorded rotation parameters to obtain the rotated 3D ultrasound volume data. Finally, the rotated target is designated as the final clipping template, and a voxel value multiplication operation is performed between it and the rotated 3D ultrasound volume data to filter out the aligned fetal head region from the 3D ultrasound volume data.

[0083] Furthermore, since interpolation operations are involved during rotation, the outline of the rotated target may not be smooth. Therefore, after obtaining the rotated target, multiple dilation and erosion operations are performed on it to obtain the final rotated target.

[0084] Based on the above embodiments, as a feasible implementation method, if the target examination object is a follicle, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object to determine the location region of the examination site corresponding to the target examination object using the three-dimensional examination network. This includes: inputting the three-dimensional ultrasound data into the three-dimensional follicle examination network to determine multiple follicle regions using the three-dimensional follicle examination network; correspondingly, determining the examination result corresponding to the target examination object based on the location region of the examination site includes: counting the volume of multiple follicle regions, sorting the multiple follicles according to their volume from smallest to largest, and displaying the volume and sorting result of the multiple follicle regions.

[0085] In practical implementation, if the target object identified by the examination object recognition network is a follicle, then the corresponding examination site is the follicle. That is, the 3D follicle examination network is used to identify multiple follicular regions in the 3D ultrasound data. Then, the 3D follicle examination network calculates the volume of these follicular regions, statistically analyzes and sorts them in ascending order of volume, and finally displays these statistical values ​​and sorting results on the ultrasound equipment so that doctors can quickly and accurately assess the number and size of follicles. In other words, if the target object identified by the examination object recognition network is a follicle, then the corresponding examination result displays the volume and sorting results of multiple follicular regions.

[0086] Based on the above embodiments, as a feasible implementation method, if the target examination object is the uterus, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object to determine the location area of ​​the examination site corresponding to the target examination object using the three-dimensional examination network. This includes: inputting the three-dimensional ultrasound data into the three-dimensional uterine examination network to determine the endometrial region using the three-dimensional uterine examination network; correspondingly, determining the examination result corresponding to the target examination object based on the location area of ​​the examination site includes: determining the target to be rotated based on the endometrial region; determining the key surface in the target to be rotated, rotating the target to be rotated to a direction where the key surface is parallel to the screen to obtain the rotated target; determining the aligned endometrial region based on the rotated target, and displaying the aligned endometrial region.

[0087] In practical implementation, if the target object identified by the examination object recognition network is the uterus, then the corresponding examination area is the endometrial region. That is, the 3D uterine examination network is used to determine voxel points belonging to the endometrial region in the 3D ultrasound data. Based on these voxel points, a target to be rotated is constructed. Key planes, i.e., the optimal viewing planes, are determined based on key information on the target. This key information can include key point locations and structural information. Then, through calculation and application of rotation transformation, the target is rotated until the key planes are parallel to the screen, resulting in the rotated target. Based on the rotated target, the aligned endometrial region is determined. The voxel values ​​of each voxel point in the aligned endometrial region are consistent with the voxel values ​​of the voxel points in the 3D ultrasound data acquired by the ultrasound equipment. The aligned endometrial region can then be displayed using the ultrasound equipment.

[0088] As a feasible implementation, determining the target to be rotated based on the endometrial region includes: directly determining the endometrial region as the target to be rotated; correspondingly, determining the aligned endometrial region based on the rotated target includes: determining the rotated target as the aligned endometrial region.

[0089] In practice, the segmented region corresponding to the endometrial region can be directly extracted from the original three-dimensional ultrasound data based on the voxel points belonging to the endometrial region as the target to be rotated. At this time, the voxel values ​​of each voxel point in the target to be rotated are consistent with the voxel values ​​of the voxel points in the original three-dimensional ultrasound data. After rotating the target to be rotated to the target direction, the rotated target is the straightened endometrial region.

[0090] As another feasible implementation, determining the target to be rotated based on the endometrial region includes: determining an initial cutting template based on the endometrial region, and determining the initial cutting template as the target to be rotated; wherein, the voxel value of the voxel point belonging to the endometrial region in the initial cutting template is 1, and the voxel value of other voxel points is 0; correspondingly, determining the key facet in the target to be rotated and rotating the target to be rotated to a direction where the key facet is parallel to the screen to obtain the rotated target includes: determining the key facet in the initial cutting template, rotating the initial cutting template to a direction where the key facet is parallel to the screen to obtain a straightened cutting template, and recording the rotation parameters during the rotation process; correspondingly, determining the straightened endometrial region based on the rotated target includes: performing a rotation operation on the three-dimensional ultrasound data based on the rotation parameters to obtain rotated three-dimensional ultrasound data; multiplying the voxel value of the voxel point in the rotated three-dimensional ultrasound data with the voxel value of the corresponding voxel point in the straightened cutting template to obtain the straightened endometrial region.

[0091] In the specific implementation, an initial clipping template is first created. In this initial clipping template, the voxel values ​​of voxel points belonging to the endometrial region are 1, while the voxel values ​​of other voxel points are 0. Then, the initial clipping template is designated as the target to be rotated. The target is rotated to the target direction to obtain the rotated target, and the rotation parameters during the rotation process are recorded. At this point, the voxel values ​​of each voxel point in the rotated target are either 1 or 0, which do not correspond to the voxel values ​​of the voxel points in the original 3D ultrasound volume data. Therefore, in order to obtain the aligned endometrial region in the 3D ultrasound volume data, the same rotation operation needs to be performed on the original 3D ultrasound volume data using the recorded rotation parameters to obtain the rotated 3D ultrasound volume data. Finally, the rotated target is designated as the final clipping template, and a voxel value multiplication operation is performed between it and the rotated 3D ultrasound volume data to filter out the aligned endometrial region from the 3D ultrasound volume data.

[0092] Furthermore, since interpolation operations are involved during rotation, the outline of the rotated target may not be smooth. Therefore, after obtaining the rotated target, multiple dilation and erosion operations are performed on it to obtain the final rotated target.

[0093] In the process of rotating the target to obtain the rotated target, the center of gravity of the target to be rotated is first used as the origin of the coordinate system, and the X, Y, and Z axes are defined according to the intersection direction of the sagittal plane, cross section and coronal plane of the target to be rotated, and a spatial coordinate system is constructed.

[0094] As a feasible implementation method, the plane with the largest projected area of ​​the target to be rotated, i.e., the key plane, is determined in the spatial coordinate system, and a normal vector perpendicular to this key plane is also determined. Simultaneously, the vectors of the corners of the target to be rotated, i.e., the vectors connecting the apexes of the two corners, also need to be determined. Finally, following the rotation sequence along the Z, X, and Y axes, the target to be rotated is adjusted through rotation operations so that the normal vector is parallel to the X-axis, and the corner vectors are parallel to the Y-axis. At this point, the key plane is parallel to the screen direction, thus achieving the standardized orientation of the target to be rotated in space, resulting in the rotated target.

[0095] As another feasible implementation method, two palace corner points and a palace bottom point are determined in the spatial coordinate system. The one with the smaller y-coordinate is the first palace corner point, and the one with the larger y-coordinate is the second palace corner point. The target to be rotated is rotated with the first palace corner point as the rotation center until the z-coordinate of the second palace corner point is equal to the z-coordinate of the first palace corner point, and the x-coordinates of the second palace corner point and the palace bottom point are all equal to the x-coordinates of the first palace corner point. At this time, the key plane is parallel to the direction of the screen, and the rotated target is obtained.

[0096] The following describes a three-dimensional ultrasound body data processing device provided in the embodiments of this application. The three-dimensional ultrasound body data processing device described below and the three-dimensional ultrasound body data processing method described above can be referred to each other.

[0097] See Figure 4 A structural diagram of a three-dimensional ultrasound body data processing device according to an exemplary embodiment is shown, as follows: Figure 4 As shown, it includes:

[0098] The first determining module 401 is used to acquire three-dimensional ultrasound body data and determine a two-dimensional ultrasound image from the three-dimensional ultrasound body data.

[0099] The second determining module 402 is used to input the two-dimensional ultrasound image into the examination object recognition network, so as to use the examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image;

[0100] The third determining module 403 is used to input the three-dimensional ultrasound body data into the three-dimensional examination network corresponding to the target examination object, so as to determine the examination result using the three-dimensional examination network.

[0101] The three-dimensional ultrasound body data processing device provided in this application embodiment automatically identifies the target examination object through an examination object recognition network, and automatically analyzes the three-dimensional ultrasound body data through the three-dimensional examination network corresponding to the target examination object to determine the examination result. Therefore, this application embodiment optimizes the examination process through automation, reduces the cumbersome operation of constantly switching application scenarios to activate different diagnostic functions, and improves the efficiency of ultrasound examination.

[0102] Based on the above embodiments, as a preferred implementation, the second determining module 402 is specifically used for: inputting the two-dimensional ultrasound image into an examination object recognition network; extracting a first semantic feature of the two-dimensional ultrasound image based on a first feature extraction branch in the examination object recognition network; wherein the first feature extraction branch includes one or more first feature extraction modules for extracting semantic features of the channel dimension; extracting a second semantic feature of the two-dimensional ultrasound image based on a second feature extraction branch in the examination object recognition network; wherein the second feature extraction branch includes one or more second feature extraction modules for extracting semantic features of the spatial dimension; fusing the first semantic feature and the second semantic feature through a pooling layer to obtain the target semantic feature of the two-dimensional ultrasound image; and determining the target examination object of the two-dimensional ultrasound image based on the target semantic feature.

[0103] Based on the above embodiments, as a preferred implementation, if the first feature extraction branch includes N first feature extraction modules and the second feature extraction branch includes N second feature extraction modules, then the input feature of the i-th second feature extraction module in the second feature extraction branch is a fusion feature of the input feature of the (i-1)-th second feature extraction module, the output feature of the (i-1)-th second feature extraction module, and the output feature of the (i-1)-th first feature extraction module. The input feature of the pooling layer includes the output feature of the N-th first feature extraction module, the output feature of the N-th second feature extraction module, and the input feature of the N-th second feature extraction module; 2≤i≤N, N≥2.

[0104] Based on the above embodiments, as a preferred implementation, the third determining module 403 is specifically used to: input the three-dimensional ultrasound body data into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location area of ​​the examination part corresponding to the target examination object; and determine the examination result corresponding to the target examination object based on the location area of ​​the examination part.

[0105] Based on the above embodiments, as a preferred implementation, if the target examination object is the pelvic floor, the third determining module 403 is specifically used to: input the three-dimensional ultrasound body data into a three-dimensional pelvic floor examination network to determine the urethral region using the three-dimensional pelvic floor examination network; determine multiple two-dimensional slices of the urethral region and determine the center points of the multiple two-dimensional slices; connect the center points of the multiple two-dimensional slices according to the arrangement order of the multiple two-dimensional slices in the urethral region to display the location of the urethra.

[0106] Based on the above embodiments, as a preferred implementation, if the target examination object is a fetus, the third determining module 403 is specifically used to: input the three-dimensional ultrasound body data into a three-dimensional fetal examination network to determine the fetal head region using the three-dimensional fetal examination network; determine the key surface of the fetal head region; rotate the fetal head region to a direction in which the key surface is parallel to the screen to obtain the aligned fetal head region; and display the aligned fetal head region.

[0107] Based on the above embodiments, as a preferred implementation, if the target object to be examined is a follicle, the third determining module 403 is specifically used to: input the three-dimensional ultrasound body data into a three-dimensional follicle examination network to determine multiple follicle regions using the three-dimensional follicle examination network; count the volume of the multiple follicle regions, sort the multiple follicles according to their volume from smallest to largest, and display the volume and sorting results of the multiple follicle regions.

[0108] Based on the above embodiments, as a preferred implementation, if the target examination object is the uterus, the third determining module 403 is specifically used for: inputting the three-dimensional ultrasound body data into a three-dimensional uterine examination network to determine the endometrial region using the three-dimensional uterine examination network; determining an initial cutting template based on the endometrial region; wherein, in the initial cutting template, the voxel value of the voxel point belonging to the endometrial region is 1, and the voxel value of other voxel points is 0; determining the key surface in the initial cutting template, rotating the initial cutting template to a direction where the key surface is parallel to the screen to obtain a corrected cutting template, and recording the rotation parameters during the rotation process; performing a rotation operation on the three-dimensional ultrasound body data based on the rotation parameters to obtain rotated three-dimensional ultrasound body data; multiplying the voxel value of the voxel point in the rotated three-dimensional ultrasound body data with the voxel value of the corresponding voxel point in the corrected cutting template to obtain the corrected endometrial region, and displaying the corrected endometrial region.

[0109] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0110] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 5 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 5 As shown, the electronic device includes:

[0111] Communication interface 1 enables information exchange with other devices, such as network devices;

[0112] Processor 2 is connected to communication interface 1 to enable information exchange with other devices. When running a computer program, it executes the three-dimensional ultrasound body data processing method provided by one or more of the above-mentioned technical solutions. The computer program is stored in memory 3.

[0113] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general will label all buses as Bus System 4.

[0114] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0115] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0116] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0117] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0118] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0119] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0120] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0121] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for processing three-dimensional ultrasound body data, characterized in that, include: Acquire three-dimensional ultrasound volume data, and determine two-dimensional ultrasound images from the three-dimensional ultrasound volume data; The two-dimensional ultrasound image is input into the examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image using the examination object recognition network; The three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to determine the examination result using the three-dimensional examination network.

2. The three-dimensional ultrasound body data processing method according to claim 1, characterized in that, Inputting the two-dimensional ultrasound image into an examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image using the examination object recognition network includes: The two-dimensional ultrasound image is input into the examination object recognition network; In the object recognition network, the first semantic features of the two-dimensional ultrasound image are extracted based on the first feature extraction branch; wherein, the first feature extraction branch includes one or more first feature extraction modules for extracting semantic features of the channel dimension; In the object recognition network, the second semantic features of the two-dimensional ultrasound image are extracted based on the second feature extraction branch; wherein, the second feature extraction branch includes one or more second feature extraction modules for extracting semantic features of the spatial dimension; The target semantic features of the two-dimensional ultrasound image are obtained by fusing the first semantic features and the second semantic features through a pooling layer; The target examination object of the two-dimensional ultrasound image is determined based on the target semantic features.

3. The three-dimensional ultrasound body data processing method according to claim 2, characterized in that, If the first feature extraction branch includes N first feature extraction modules and the second feature extraction branch includes N second feature extraction modules, then the input feature of the i-th second feature extraction module in the second feature extraction branch is a fusion feature of the input feature of the (i-1)-th second feature extraction module, the output feature of the (i-1)-th second feature extraction module, and the output feature of the (i-1)-th first feature extraction module. The input feature of the pooling layer includes the output feature of the N-th first feature extraction module, the output feature of the N-th second feature extraction module, and the output feature of the (N-1)-th second feature extraction module. 2≤i≤N, N≥2.

4. The three-dimensional ultrasound body data processing method according to claim 1, characterized in that, The three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target object to determine the examination results using the three-dimensional examination network, including: The three-dimensional ultrasound body data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location area of ​​the examination site corresponding to the target examination object; The inspection result corresponding to the target inspection object is determined based on the location area of ​​the inspection site.

5. The three-dimensional ultrasound body data processing method according to claim 4, characterized in that, If the target examination object is the pelvic floor, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location area of ​​the examination site corresponding to the target examination object, including: The three-dimensional ultrasound data is input into a three-dimensional pelvic floor examination network to determine the urethral region using the three-dimensional pelvic floor examination network. Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes: Determine multiple two-dimensional slices of the urethral region, and determine the center point of each of the multiple two-dimensional slices; The center points of the multiple two-dimensional slices are connected according to their arrangement order in the urethral region to show the location of the urethra.

6. The three-dimensional ultrasound body data processing method according to claim 4, characterized in that, If the target examination object is a fetus, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location region of the examination site corresponding to the target examination object, including: The three-dimensional ultrasound data is input into a three-dimensional fetal examination network to determine the fetal head region using the three-dimensional fetal examination network; Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes: Determine the key plane of the fetal head region, rotate the fetal head region so that the key plane is parallel to the screen to obtain the corrected fetal head region, and display the corrected fetal head region.

7. The three-dimensional ultrasound body data processing method according to claim 4, characterized in that, If the target object for examination is a follicle, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target object to determine the location region of the examination site corresponding to the target object using the three-dimensional examination network, including: The three-dimensional ultrasound data is input into a three-dimensional follicle detection network to determine multiple follicle regions. Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes: The volumes of multiple follicular regions are counted, and the multiple follicles are sorted from smallest to largest volume. The volumes and sorting results of the multiple follicular regions are then displayed.

8. The three-dimensional ultrasound body data processing method according to claim 4, characterized in that, If the target examination object is the uterus, the three-dimensional ultrasound data is input into the three-dimensional examination network corresponding to the target examination object, so as to use the three-dimensional examination network to determine the location area of ​​the examination site corresponding to the target examination object, including: The three-dimensional ultrasound data is input into a three-dimensional uterine examination network to determine the endometrial region using the three-dimensional uterine examination network; Accordingly, determining the inspection result corresponding to the target inspection object based on the location area of ​​the inspection site includes: An initial cutting template is determined based on the endometrial region; wherein, the voxel value of the voxel point belonging to the endometrial region in the initial cutting template is 1, and the voxel value of other voxel points is 0; Determine the key surfaces in the initial cutting template, rotate the initial cutting template until the key surfaces are parallel to the screen to obtain the aligned cutting template, and record the rotation parameters during the rotation process; Based on the rotation parameters, the three-dimensional ultrasound body data is rotated to obtain rotated three-dimensional ultrasound body data; The voxel values ​​of the voxel points in the rotated three-dimensional ultrasound data are multiplied with the voxel values ​​of the corresponding voxel points in the aligned cutting template to obtain the aligned endometrial region, and the aligned endometrial region is displayed.

9. A three-dimensional ultrasound body data processing device, characterized in that, include: The first determining module is used to acquire three-dimensional ultrasound volume data and determine a two-dimensional ultrasound image from the three-dimensional ultrasound volume data. The second determining module is used to input the two-dimensional ultrasound image into the examination object recognition network, so as to use the examination object recognition network to determine the target examination object corresponding to the two-dimensional ultrasound image; The third determining module is used to input the three-dimensional ultrasound body data into the three-dimensional examination network corresponding to the target examination object, so as to determine the examination result using the three-dimensional examination network.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the three-dimensional ultrasound body data processing method as described in any one of claims 1 to 8.

11. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a computer program that, when executed, implements the steps of the three-dimensional ultrasound body data processing method as described in any one of claims 1 to 8.

Images