Abdominal aorta navigation method, electronic equipment and storage medium
Through the combination of B-ultrasound image and abdominal aortic navigation model, the problem of primary medical personnel positioning abdominal aortic bifurcation during B-ultrasound examinations was solved, achieving fast and accurate navigation and time savings.
Patent Information
- Application Number
- CN202411362330.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2024-12-31
AI Technical Summary
The lack of a scheme for navigating the abdominal aorta based on B-ultrasound images in the prior art has made it difficult for primary medical personnel to accurately locate the abdominal aorta bifurcation when conducting B-ultrasound examinations, which increases the difficulty and time of examination.
By obtaining the B-ultrasound image obtained by ultrasound scan and the pre-trained abdominal aorta navigation model, including the abdominal aorta segmentation head and bifurcation judgment head, the image is input to the model to generate the segmentation feature map and the judgment feature map, and according to the feature map The results of abdominal aorta segmentation and bifurcation judgment are obtained, and prompt information is generated when the conditions are met to guide the scanner to quickly find the abdominal aorta bifurcation.
The abdominal aorta navigation based on B-ultrasound images is realized, helping scanners quickly locate the abdominal aorta bifurcation, saving examination time and reducing experience and technical requirements.
Smart Images
Figure CN119214688A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of image processing, and in particular to an abdominal aorta navigation method, electronic equipment and storage medium. Background Art
[0002] B-ultrasound is an important means of medical examination. Its biggest difference from other imaging equipment is that it is light and easy to carry, so it is easy to promote to primary medical institutions.
[0003] However, for primary care personnel who do not have rich experience and professional skills, it is a challenge to perform accurate B-ultrasound examinations. In response to this, solutions for navigation based on B-ultrasound images have emerged, which can assist relevant personnel in performing B-ultrasound examinations, thereby reducing the requirements for experience and skills.
[0004] After investigation, it was found that there is currently no relevant solution for abdominal aorta navigation based on B-ultrasound images. Summary of the invention
[0005] The embodiments of the present invention provide an abdominal aorta navigation method, an electronic device and a storage medium to implement abdominal aorta navigation based on B-ultrasound images.
[0006] According to one aspect of the present invention, there is provided an abdominal aorta navigation method, which may include:
[0007] Acquire a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation judgment head;
[0008] Input the B-ultrasound image into the abdominal aorta navigation model to obtain the segmentation feature map output by the abdominal aorta segmentation head and the judgment feature map output by the bifurcation judgment head;
[0009] According to the segmentation feature map, the abdominal aorta segmentation result is obtained, and according to the judgment feature map, the bifurcation judgment result is obtained;
[0010] When the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation determination result indicates that the B-ultrasound image contains a bifurcation, prompt information is generated and displayed, wherein the prompt information indicates that the abdominal aorta bifurcation has been currently scanned.
[0011] According to another aspect of the present invention, there is provided an electronic device, which may include:
[0012] at least one processor; and
[0013] a memory communicatively connected to at least one processor; wherein,
[0014] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor implements the abdominal aorta navigation method provided by any embodiment of the present invention when executing the computer program.
[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a processor to implement the abdominal aorta navigation method provided by any embodiment of the present invention when executed.
[0016] The technical solution of the embodiment of the present invention is to obtain a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation judgment head; input the B-ultrasound image into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head and a judgment feature map output by the bifurcation judgment head; obtain an abdominal aorta segmentation result according to the segmentation feature map, and obtain a bifurcation judgment result according to the judgment feature map; when the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation judgment result indicates that the B-ultrasound image contains a bifurcation, this indicates that the bifurcation of the abdominal aorta has been scanned, and a prompt message indicating that the bifurcation of the abdominal aorta has been scanned can be generated and displayed, thereby prompting the scanning personnel that the current scanning task can be ended. The above technical solution can realize abdominal aorta navigation based on B-ultrasound images, which helps to guide the scanning personnel to quickly scan the target position (i.e., the position of the bifurcation of the abdominal aorta), thereby saving scanning time.
[0017] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0019] Figure 1 is a flow chart of an abdominal aorta navigation method provided according to an embodiment of the present invention;
[0020] Figure 2 is a schematic diagram of a model structure in an abdominal aorta navigation method provided by an embodiment of the present invention;
[0021] Figure 3 is a flow chart of another abdominal aorta navigation method provided according to an embodiment of the present invention;
[0022] Figure 4 is a schematic diagram showing the comparison of the segmentation contour before and after Gaussian filtering in another abdominal aorta navigation method provided by an embodiment of the present invention;
[0023] Figure 5 is a flow chart of another abdominal aorta navigation method provided according to an embodiment of the present invention;
[0024] Figure 6 is a structural block diagram of an abdominal aorta navigation device provided according to an embodiment of the present invention;
[0025] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the abdominal aorta navigation method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0028] Figure 1 This is a flow chart of an abdominal aorta navigation method provided in an embodiment of the present invention. This embodiment is applicable to the case of abdominal aorta navigation, and is particularly applicable to the case of abdominal aorta navigation based on B-ultrasound images. The method can be performed by an abdominal aorta navigation device provided in an embodiment of the present invention, which can be implemented by software and / or hardware, and can be integrated in an electronic device, which can be various user terminals or servers.
[0029] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:
[0030] S110. Acquire a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation determination head.
[0031] The B-ultrasound image can be understood as an image obtained by scanning the target part of the target object with B-ultrasound. It can be understood that when the scanning personnel moves the probe over the target part, the B-ultrasound image is updated along with the movement of the probe. Obtain the B-ultrasound image.
[0032] The abdominal aorta navigation model can be understood as a pre-trained deep learning model for abdominal aorta navigation, and can be further understood as a deep learning model for realizing abdominal aorta navigation through abdominal aorta segmentation and bifurcation judgment. The abdominal aorta navigation model can include an abdominal aorta segmentation head and a bifurcation judgment head. The abdominal aorta segmentation head can be understood as a module for performing abdominal aorta segmentation on B-ultrasound images, and the bifurcation judgment head can be understood as a module for judging whether there is a bifurcation in the B-ultrasound image. The bifurcation may be an abdominal aorta bifurcation or other bifurcation, which is related to the actual situation and is not limited here. Obtain the abdominal aorta navigation model.
[0033] S120. Input the B-ultrasound image into the abdominal aorta navigation model to obtain the segmentation feature map output by the abdominal aorta segmentation head and the judgment feature map output by the bifurcation judgment head.
[0034] The segmentation feature map can be understood as a feature map output by the abdominal aorta segmentation head and used to segment the abdominal aorta for the B-ultrasound image. The judgment feature map can be understood as a feature map output by the bifurcation judgment head and used to judge whether there is a bifurcation in the B-ultrasound image.
[0035] The B-ultrasound image is input into the abdominal aorta navigation model to obtain a segmentation feature map and a judgment feature map.
[0036] S130. Obtain an abdominal aorta segmentation result based on the segmentation feature map, and obtain a bifurcation judgment result based on the judgment feature map.
[0037] The abdominal aorta segmentation result can be understood as the result obtained after performing abdominal aorta segmentation on the B-ultrasound image, which can indicate whether the abdominal aorta exists in the B-ultrasound image, that is, whether the abdominal aorta in the target part is currently scanned. The abdominal aorta segmentation result is obtained according to the segmentation feature map.
[0038] In practical applications, optionally, after obtaining the abdominal aorta segmentation result, the abdominal aorta segmentation result can be displayed to assist the scanner to determine the current scanned position, and then adjust the placement position and movement direction of the probe based on this to facilitate moving toward the target position (i.e., the position of the abdominal aorta bifurcation).
[0039] The bifurcation judgment result can be understood as the result obtained after bifurcation judgment on the B-ultrasound image, which can characterize whether there is a bifurcation in the B-ultrasound image, that is, whether the bifurcation in the target part is currently scanned. The bifurcation judgment result is obtained according to the judgment feature map.
[0040] S140. When the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation determination result indicates that the B-ultrasound image contains a bifurcation, generate and display prompt information, wherein the prompt information indicates that the abdominal aorta bifurcation has been scanned.
[0041] Among them, when the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation judgment result indicates that the B-ultrasound image contains a bifurcation, it means that the abdominal aorta bifurcation has been scanned, that is, the current scanned position is the position of the abdominal aorta bifurcation. Then, a prompt information indicating that the abdominal aorta bifurcation has been scanned can be generated and displayed to remind the scanning personnel that the current scanning task can be completed.
[0042] At this point, abdominal aorta navigation is completed.
[0043] The technical solution of the embodiment of the present invention is to obtain a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation judgment head; input the B-ultrasound image into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head and a judgment feature map output by the bifurcation judgment head; obtain an abdominal aorta segmentation result according to the segmentation feature map, and obtain a bifurcation judgment result according to the judgment feature map; when the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation judgment result indicates that the B-ultrasound image contains a bifurcation, this indicates that the bifurcation of the abdominal aorta has been scanned, and a prompt message indicating that the bifurcation of the abdominal aorta has been scanned can be generated and displayed, thereby prompting the scanning personnel that the current scanning task can be ended. The above technical solution can realize abdominal aorta navigation based on B-ultrasound images, which helps to guide the scanning personnel to quickly scan the target position (i.e., the position of the bifurcation of the abdominal aorta), thereby saving scanning time.
[0044] In an optional technical solution, the abdominal aorta navigation model further includes a first encoding module, which inputs the B-ultrasound image into the abdominal aorta navigation model to obtain a judgment feature map output by the bifurcation judgment head, including:
[0045] Inputting the B-ultrasound image into the abdominal aorta navigation model, encoding the B-ultrasound image by using a first encoding module to obtain a first feature map, and performing a full connection operation based on the first feature map by using a bifurcated judgment head to obtain a judgment feature map;
[0046] Get the judgment feature map output by the bifurcation judgment head.
[0047] Among them, the first encoding module can be understood as the module responsible for encoding in the abdominal aorta navigation model. In combination with the application scenarios that may be involved in the embodiments of the present invention, the module can be implemented based on the convolution layer and the downsampling layer. After the B-ultrasound image is input into the abdominal aorta navigation model, the first encoding module can be used to encode the B-ultrasound image to obtain a first feature map; further, the bifurcation judgment head can be used to perform a full connection operation based on the first feature map to obtain a judgment feature map, thereby obtaining a judgment feature map output by the bifurcation judgment head.
[0048] The above technical solution realizes accurate extraction of the judgment feature map through encoding and full connection operations.
[0049] In order to better understand the above technical solution, the following is an exemplary description of it in combination with specific examples. Figure 2 The abdominal aorta navigation model includes an encoding module (i.e., the first encoding module described above), a bifurcation judgment head, and an abdominal aorta segmentation head (the unframed part in the figure is the abdominal aorta segmentation head). The encoding module is a lightweight backbone network, whose input is a 256×256 B-ultrasound image. After continuous depth-separable convolutional layers and downsampling layers, 128×128, 64×64, and 32×32 feature maps can be obtained. The 32×32 feature map is the first feature map described above. The bifurcation judgment head is implemented based on a fully connected layer. By flattening the first feature map and then performing continuous full-connection operations on the obtained flattened feature map, a judgment feature map can be obtained.
[0050] Before introducing the following embodiments, the application scenarios thereof are first exemplified. For example, in order to improve the convenience of abdominal aorta navigation application, a lightweight abdominal aorta navigation model can be designed, and then the abdominal aorta navigation model can be deployed on a mobile device (such as a mobile phone and / or a tablet, etc.) to run.
[0051] On this basis, due to the limited computing resources of mobile devices, there are some differences between the training stage and the application stage of the model. Specifically, the size of the segmentation feature map output by the model in the training stage is the same as the size of the B-ultrasound image, while the size of the segmentation feature map output in the application stage is smaller than the size of the B-ultrasound image. In this application scenario, the abdominal aorta segmentation result is obtained through the following example.
[0052] Figure 3 It is a flow chart of another abdominal aorta navigation method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the first size of the segmentation feature map is smaller than the second size of the B-ultrasound image; according to the segmentation feature map, the abdominal aorta segmentation result is obtained, including: according to the segmentation feature map, a preliminary segmentation result is obtained; a segmentation contour is extracted from the preliminary segmentation result, and the segmentation contour is upsampled according to the first size and the second size, so as to obtain the abdominal aorta segmentation result based on the upsampled contour. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.
[0053] See also Figure 3 The method of this embodiment may specifically include the following steps:
[0054] S210. Acquire a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation determination head.
[0055] S220. Input the B-ultrasound image into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head and a judgment feature map output by the bifurcation judgment head, wherein a first size of the segmentation feature map is smaller than a second size of the B-ultrasound image.
[0056] Among them, the first size can be understood as the size of the segmentation feature map, and the second size can be understood as the size of the B-ultrasound image. The first size is smaller than the second size, which helps to reduce the computing resources required for the model operation.
[0057] S230. Obtain a preliminary segmentation result based on the segmentation feature map.
[0058] The preliminary segmentation result can be understood as a preliminary segmentation result for the abdominal aorta, and the size of the segmentation result is the first size. The preliminary segmentation result is obtained according to the segmentation feature map.
[0059] Exemplarily, sigmoid mapping can be performed on the segmentation feature map, and then the pixels whose mapping values are greater than a preset numerical threshold are regarded as the abdominal aorta, and the pixels whose mapping values are less than or equal to the preset numerical threshold are regarded as the background, thereby obtaining a preliminary segmentation result.
[0060] S240. Extract a segmentation contour from the preliminary segmentation result, and upsample the segmentation contour according to the first size and the second size to obtain an abdominal aorta segmentation result based on the upsampled contour.
[0061] Among them, a segmentation contour is extracted from the preliminary segmentation result, and the segmentation contour can be understood as the contour of the abdominal aorta in the preliminary segmentation result. It should be noted that the size of the B-ultrasound image input into the abdominal aorta navigation model is the second size, which means that an abdominal aorta segmentation result of the second size needs to be obtained, but the size corresponding to the segmentation contour is the first size, so the segmentation contour can be upsampled according to the numerical relationship between the first size and the second size to obtain an upsampled contour corresponding to the second size, and then the abdominal aorta segmentation result is obtained based on the upsampled contour, especially based on the upsampled contour and the B-ultrasound image.
[0062] For example, in Figure 2 In the example shown, the second size may be 256×256 and the first size may be 128×128, so the segmented contour may be upsampled by a factor of 2 to obtain an upsampled contour.
[0063] S250. Obtain a bifurcation judgment result based on the judgment feature graph.
[0064] S260. When the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation determination result indicates that the B-ultrasound image contains a bifurcation, generate and display prompt information, wherein the prompt information indicates that the abdominal aorta bifurcation has been scanned.
[0065] The technical solution of the embodiment of the present invention, compared with extracting the segmentation contour using the preliminary segmentation result of the second size (such as the preliminary segmentation result obtained according to the segmentation feature map of the second size), can save a lot of computing resources and be compatible with mobile devices by extracting the segmentation contour using the preliminary segmentation result of the first size.
[0066] In an optional technical solution, after extracting the segmentation contour from the preliminary segmentation result, the above abdominal aorta navigation method further includes:
[0067] For all the pixel points on the segmentation contour, the horizontal coordinates of all the pixel points are filtered, and the vertical coordinates of all the pixel points are filtered;
[0068] Based on the coordinate filtering results, the segmentation contour is updated.
[0069] Among them, it is found through practice that the segmentation contour extracted using the preliminary segmentation result of the first size is jagged compared to the segmentation contour extracted using the preliminary segmentation result of the second size, which will affect the accuracy of the abdominal aorta segmentation. In order to achieve accurate segmentation of the abdominal aorta, after extracting the segmentation contour, the horizontal coordinates and vertical coordinates of all pixel points on the segmentation contour can be filtered respectively, thereby achieving the effect of smoothing the segmentation contour and thus improving the accuracy of the abdominal aorta segmentation.
[0070] For example, Figure 4 The left side shows a segmentation contour 1 extracted using the preliminary segmentation result of the first size, and the segmentation contour 1 is jagged; Figure 4 The right side shows the segmentation contour 2 after filtering by a 1-dimensional Gaussian filter. Compared with the segmentation contour 1, the segmentation contour 2 is smoother.
[0071] As another optional technical solution, the abdominal aorta segmentation head is pre-trained by the following steps:
[0072] Acquire a sample image obtained through B-ultrasound scanning and an abdominal aorta segmentation label of the sample image;
[0073] Inputting the sample image into the original navigation model to obtain a sample feature map output by the original segmentation head in the original navigation model, wherein the size of the sample feature map is the second size;
[0074] According to the sample feature map, the sample segmentation result is obtained, and based on the sample segmentation result and the abdominal aorta segmentation label, the parameters in the original segmentation head are adjusted to obtain the abdominal aorta segmentation head.
[0075] Among them, since the model training process is carried out on a fixed device with sufficient computing resources, the original segmentation head corresponding to the abdominal aorta segmentation head can output a sample feature map with a size of the second size, and then based on the sample feature map, a sample segmentation result with a size of the second size can be obtained. The sample segmentation result can be understood as the abdominal aorta segmentation result in the model training process, and then the original segmentation head can be trained based on the sample segmentation result and the corresponding abdominal aorta segmentation label to obtain the abdominal aorta segmentation head.
[0076] Figure 5This is a flow chart of another abdominal aorta navigation method provided in an embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the abdominal aorta navigation model also includes a second encoding module, the second encoding module includes at least two convolutional layers; the B-ultrasound image is input into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head, including: inputting the B-ultrasound image into the abdominal aorta navigation model to encode the B-ultrasound image using the second encoding module to obtain a second feature map, and using the abdominal aorta segmentation head, in combination with the second feature map, to perform the following three steps: performing an upsampling operation related to the second feature map to obtain an upsampling feature map, wherein the size of the upsampling feature map is the first size; cascading the upsampling feature map with a size feature map of the first size in the convolutional feature maps output by at least two convolutional layers to obtain a cascaded feature map, wherein the convolutional layer that outputs the size feature map is between the first convolutional layer and the last convolutional layer, and the convolutional feature map output by the last convolutional layer is the second feature map; performing a convolution operation on the cascaded feature map to obtain a segmentation feature map; and obtaining the segmentation feature map output by the abdominal aorta segmentation head. The explanations of terms that are the same as or corresponding to the above embodiments are not repeated here.
[0077] See also Figure 5 The method of this embodiment may specifically include the following steps:
[0078] S310. Obtain a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes a second encoding module, an abdominal aorta segmentation head and a bifurcation judgment head, and the second encoding module includes at least two convolutional layers.
[0079] The second encoding module can be understood as the module responsible for encoding in the abdominal aorta navigation model. In combination with the application scenarios that may be involved in the embodiments of the present invention, the module can be the same as or different from the first encoding module described above. This can be set according to actual needs and is not specifically limited here. Figure 2 In the example given, both are the same encoding module.
[0080] S320. Input the B-ultrasound image into the abdominal aorta navigation model, use the second encoding module to encode the B-ultrasound image to obtain a second feature map, and use the abdominal aorta segmentation head to perform the following S330-S340 in combination with the second feature map to obtain a segmentation feature map.
[0081] After the B-ultrasound image is input into the abdominal aorta navigation model, the second encoding module can be used to encode the B-ultrasound image to obtain a second feature map; further, the abdominal aorta segmentation head can be used to perform the following two steps in combination with the second feature map to obtain a segmentation feature map. On this basis, optionally, a bifurcation judgment head can be used to obtain a judgment feature map based on the second feature map.
[0082] S330. Perform an upsampling operation related to the second feature map to obtain an upsampled feature map, wherein a first size of the upsampled feature map is smaller than a second size of the B-ultrasound image.
[0083] Among them, the size of the second feature map is smaller than the first size. In this step, an upsampling operation related to the second feature map is performed to obtain an upsampled feature map with a size of the first size, so as to obtain a segmentation feature map with a size of the first size based on the upsampled feature map.
[0084] In practical applications, optionally, the upsampled feature map can be obtained in the following manner: using different convolution kernels to process the second feature map respectively, and then cascading the obtained processed feature maps to obtain a cascade result; performing an upsampling operation related to the cascade result to obtain an upsampled feature map. Here, by using different convolution kernels to process the second feature map respectively, multi-faceted feature extraction for the second feature map is achieved, and then by cascading the processed feature maps, better feature fusion can be achieved, and based on this, an upsampling operation is performed to obtain an upsampled feature map, thereby improving the richness of the segmentation feature map obtained based on the upsampled feature map.
[0085] S340. Cascade the upsampled feature map with a size feature map of the first size in the convolution feature maps respectively output by at least two convolutional layers to obtain a cascaded feature map, and perform a convolution operation on the cascaded feature map to obtain a segmentation feature map, wherein the convolution layer that outputs the size feature map is located between the first convolution layer and the last convolution layer in all the convolution layers, and the convolution feature map output by the last convolution layer is the second feature map.
[0086] Among them, the convolution layer can be understood as the convolution layer in the second encoding module, and the feature map processed by the convolution layer is referred to as the convolution feature map. Since the size of the upsampled feature map is the first size, the size feature map with the size of the first size in the convolution feature map output by each convolution layer can be cascaded with the upsampled feature map, so as to better fuse the features, and then the convolution operation is performed on the obtained cascade feature map to obtain the segmentation feature map.
[0087] It can be understood that since the size of the size feature map is smaller than the second size and larger than the size of the second feature map, and the convolution feature map output by the last convolution layer in all convolution layers is the second feature map and the size of the convolution feature map output by the first convolution layer is the second size, the convolution layer that outputs the size feature map is located between the first convolution layer and the last convolution layer in all convolution layers.
[0088] For example, see Figure 2 The abdominal aorta segmentation head processes the second feature map using 5 different convolution kernels respectively, and then cascades the obtained 5 processed feature maps, and then performs 4 times upsampling bilinear interpolation on the obtained cascade feature map to obtain a 128×128 upsampled feature map. On this basis, in order to fuse better features, the 128×128 convolution feature map in the encoding stage is also cascaded, and then the two features are completely fused through the convolution operation to obtain the segmentation feature map.
[0089] S350. Obtain a segmentation feature map output by the abdominal aorta segmentation head, and obtain a preliminary segmentation result based on the segmentation feature map.
[0090] S360. Extract a segmentation contour from the preliminary segmentation result, and upsample the segmentation contour according to the first size and the second size to obtain an abdominal aorta segmentation result based on the upsampled contour.
[0091] S370. Obtain a judgment feature graph output by the bifurcation judgment head, and obtain a bifurcation judgment result based on the judgment feature graph.
[0092] S380. When the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation determination result indicates that the B-ultrasound image contains a bifurcation, generate and display prompt information, wherein the prompt information indicates that the abdominal aorta bifurcation has been scanned.
[0093] The technical solution of the embodiment of the present invention improves the richness of the feature information conveyed by the final segmentation feature map through cascading between feature maps, thereby improving the accuracy of abdominal aorta segmentation.
[0094] Figure 6 This is a structural block diagram of an abdominal aorta navigation device provided in an embodiment of the present invention. The device can be used to execute the abdominal aorta navigation method provided in any of the above embodiments. The device and the abdominal aorta navigation method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the abdominal aorta navigation device, please refer to the embodiments of the above abdominal aorta navigation method. Figure 6 The device may specifically include: an abdominal aorta navigation model acquisition module 410, a feature map determination acquisition module 420, a bifurcation determination result acquisition module 430, and a prompt information display module 440.
[0095] The abdominal aorta navigation model acquisition module 410 is used to acquire the B-ultrasound image obtained by B-ultrasound scanning and the pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation judgment head;
[0096] The judgment feature map obtaining module 420 is used to input the B-ultrasound image into the abdominal aorta navigation model to obtain the segmentation feature map output by the abdominal aorta segmentation head and the judgment feature map output by the bifurcation judgment head;
[0097] A bifurcation judgment result obtaining module 430 is used to obtain an abdominal aorta segmentation result according to the segmentation feature map, and to obtain a bifurcation judgment result according to the judgment feature map;
[0098] The prompt information display module 440 is used to generate and display prompt information when the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation judgment result indicates that the B-ultrasound image contains a bifurcation, wherein the prompt information indicates that the abdominal aorta bifurcation has been scanned.
[0099] Optionally, the abdominal aorta navigation model further includes a first encoding module, and the feature map determination module 420 may include:
[0100] A first B-ultrasound image input unit is used to input the B-ultrasound image into the abdominal aorta navigation model, so as to encode the B-ultrasound image by using a first encoding module to obtain a first feature map, and to perform a full connection operation based on the first feature map by using a bifurcated judgment head to obtain a judgment feature map;
[0101] The judgment feature map obtaining unit is used to obtain the judgment feature map output by the bifurcation judgment head.
[0102] Optionally, the first size of the segmentation feature map is smaller than the second size of the B-ultrasound image;
[0103] The bifurcation determination result obtaining module 430 may include:
[0104] A preliminary segmentation result obtaining submodule is used to obtain a preliminary segmentation result according to the segmentation feature map;
[0105] The abdominal aorta segmentation result obtaining submodule is used to extract a segmentation contour from the preliminary segmentation result, and upsample the segmentation contour according to the first size and the second size to obtain the abdominal aorta segmentation result based on the upsampled contour.
[0106] On this basis, an optional abdominal aorta segmentation result obtaining submodule is also used for:
[0107] After extracting the segmentation contour from the preliminary segmentation result, for all the pixel points on the segmentation contour, filtering the horizontal coordinates of all the pixel points, and filtering the vertical coordinates of all the pixel points;
[0108] Based on the coordinate filtering results, the segmentation contour is updated.
[0109] Alternatively, the abdominal aorta segmentation head is pre-trained using the following modules:
[0110] An abdominal aorta segmentation label acquisition module is used to acquire a sample image obtained by B-ultrasound scanning and an abdominal aorta segmentation label of the sample image;
[0111] A sample feature map obtaining module is used to input the sample image into the original navigation model to obtain a sample feature map output by the original segmentation head in the original navigation model, wherein the size of the sample feature map is the second size;
[0112] The training module is used to obtain a sample segmentation result according to the sample feature map, and adjust the parameters in the original segmentation head based on the sample segmentation result and the abdominal aorta segmentation label to obtain the abdominal aorta segmentation head.
[0113] Alternatively, the abdominal aorta navigation model further includes a second encoding module, wherein the second encoding module includes at least two convolutional layers; the feature map determination module 420 may include:
[0114] The second B-ultrasound image input unit is used to input the B-ultrasound image into the abdominal aorta navigation model, so as to encode the B-ultrasound image by using the second encoding module to obtain a second feature map, and use the abdominal aorta segmentation head and the second feature map to run the following three subunits:
[0115] An up-sampling feature map obtaining subunit is used to perform an up-sampling operation related to the second feature map to obtain an up-sampling feature map, wherein the size of the up-sampling feature map is the first size;
[0116] A cascade feature map obtaining subunit, used for cascading the up-sampled feature map with a size feature map of a first size in the convolution feature maps respectively output by at least two convolution layers, to obtain a cascade feature map, wherein the convolution layer outputting the size feature map is between the first convolution layer and the last convolution layer, and the convolution feature map output by the last convolution layer is the second feature map;
[0117] The segmentation feature map is used to obtain a sub-unit, which is used to perform a convolution operation on the cascade feature map to obtain a segmentation feature map;
[0118] The segmentation feature map obtaining unit is used to obtain the segmentation feature map output by the abdominal aorta segmentation head.
[0119] On this basis, optionally, the feature map is upsampled to obtain a subunit, which is specifically used for:
[0120] The second feature maps are processed respectively using different convolution kernels, and the processed feature maps are cascaded to obtain a cascade result;
[0121] An upsampling operation related to the cascade result is performed to obtain an upsampled feature map.
[0122] Based on any of the above devices, optionally, the device may further include:
[0123] The abdominal aorta segmentation result display module is used to display the abdominal aorta segmentation result after obtaining the abdominal aorta segmentation result according to the segmentation feature map.
[0124] The abdominal aorta navigation device provided by the embodiment of the present invention obtains a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model through an abdominal aorta navigation model acquisition module, and the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation judgment head; through a judgment feature map acquisition module, the B-ultrasound image is input into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head and a judgment feature map output by the bifurcation judgment head; through a bifurcation judgment result acquisition module, an abdominal aorta segmentation result is obtained according to the segmentation feature map, and a bifurcation judgment result is obtained according to the judgment feature map; through a prompt information display module, when the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation judgment result indicates that the B-ultrasound image contains a bifurcation, this indicates that the bifurcation of the abdominal aorta has been scanned, then prompt information indicating that the bifurcation of the abdominal aorta has been scanned can be generated and displayed, thereby prompting the scanning personnel that the current scanning task can be ended. The above device can realize abdominal aorta navigation based on B-ultrasound images, which helps to guide the scanning personnel to quickly scan to the target position (i.e. the position of the bifurcation of the abdominal aorta), thereby saving scanning time.
[0125] The abdominal aorta navigation device provided in the embodiment of the present invention can execute the abdominal aorta navigation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0126] It is worth noting that in the above-mentioned embodiment of the abdominal aorta navigation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0127] Figure 7A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0128] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0129] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0130] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the abdominal aorta navigation method.
[0131] In some embodiments, the abdominal aorta navigation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the abdominal aorta navigation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the abdominal aorta navigation method in any other appropriate manner (e.g., by means of firmware).
[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0133] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0134] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0135] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0136] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0137] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0138] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0139] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. An abdominal aorta navigation method, characterized in that: include: Acquire a B-ultrasound image obtained by B-ultrasound scanning and a pre-trained abdominal aorta navigation model, wherein the abdominal aorta navigation model includes an abdominal aorta segmentation head and a bifurcation judgment head; Inputting the B-ultrasound image into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head and a judgment feature map output by the bifurcation judgment head; Obtaining an abdominal aorta segmentation result according to the segmentation feature map, and obtaining a bifurcation judgment result according to the judgment feature map; When the abdominal aorta segmentation result indicates that the B-ultrasound image contains the abdominal aorta and the bifurcation determination result indicates that the B-ultrasound image contains a bifurcation, prompt information is generated and displayed, wherein the prompt information indicates that the abdominal aorta bifurcation has been currently scanned.
2. The method according to claim 1, characterized in that The abdominal aorta navigation model further includes a first encoding module, wherein the B-ultrasound image is input into the abdominal aorta navigation model to obtain a judgment feature map output by the bifurcation judgment head, including: Inputting the B-ultrasound image into the abdominal aorta navigation model, encoding the B-ultrasound image by using the first encoding module to obtain a first feature map, and performing a full connection operation based on the first feature map by using the bifurcation judgment head to obtain a judgment feature map; The judgment feature graph output by the bifurcation judgment head is obtained.
3. The method according to claim 1, characterized in that The first size of the segmentation feature map is smaller than the second size of the B-ultrasound image; The step of obtaining the abdominal aorta segmentation result according to the segmentation feature map includes: Obtaining a preliminary segmentation result according to the segmentation feature map; A segmentation contour is extracted from the preliminary segmentation result, and the segmentation contour is upsampled according to the first size and the second size to obtain an abdominal aorta segmentation result based on the upsampled contour.
4. The method according to claim 3, characterized in that After extracting the segmentation contour from the preliminary segmentation result, the method further includes: For all the pixel points on the segmentation contour, filtering the horizontal coordinates of all the pixel points, and filtering the vertical coordinates of all the pixel points; Based on the coordinate filtering result, the segmentation contour is updated.
5. The method according to claim 3, characterized in that: The abdominal aorta segmentation head is pre-trained by the following steps: Acquire a sample image obtained by the B-ultrasound scan and an abdominal aorta segmentation label of the sample image; Inputting the sample image into an original navigation model to obtain a sample feature map output by an original segmentation head in the original navigation model, wherein a size of the sample feature map is the second size; A sample segmentation result is obtained according to the sample feature map, and based on the sample segmentation result and the abdominal aorta segmentation label, the parameters in the original segmentation head are adjusted to obtain the abdominal aorta segmentation head.
6. The method according to claim 3, characterized in that The abdominal aorta navigation model further includes a second encoding module, wherein the second encoding module includes at least two convolutional layers; The step of inputting the B-ultrasound image into the abdominal aorta navigation model to obtain a segmentation feature map output by the abdominal aorta segmentation head includes: The B-ultrasound image is input into the abdominal aorta navigation model, and the B-ultrasound image is encoded by using the second encoding module to obtain a second feature map, and the abdominal aorta segmentation head is used in combination with the second feature map to perform the following three steps: Performing an upsampling operation related to the second feature map to obtain an upsampled feature map, wherein a size of the upsampled feature map is the first size; Cascading the up-sampled feature map with a size feature map of the first size in the convolution feature maps respectively output by at least two of the convolution layers to obtain a cascaded feature map, wherein the convolution layer that outputs the size feature map is between the first convolution layer and the last convolution layer, and the convolution feature map output by the last convolution layer is the second feature map; Performing a convolution operation on the cascade feature map to obtain a segmentation feature map; The segmentation feature map output by the abdominal aorta segmentation head is obtained.
7. The method according to claim 6, characterized in that The performing an upsampling operation related to the second feature map to obtain an upsampled feature map includes: Using different convolution kernels to process the second feature maps respectively, and cascading the obtained processed feature maps to obtain a cascade result; An upsampling operation related to the cascade result is performed to obtain an upsampling feature map.
8. The method according to any one of claims 1 to 7, characterized in that After obtaining the abdominal aorta segmentation result according to the segmentation feature map, the method further includes: The abdominal aorta segmentation result is shown.
9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the abdominal aorta navigation method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the abdominal aorta navigation method as described in any one of claims 1-8 when executed.