A spatial guidance method for ultrasound images and related apparatus

Abstract

The application discloses a spatial guidance method for an ultrasound image and a related device. The method comprises the following steps: acquiring an ultrasound image to be positioned, and determining a target anatomical region corresponding to the ultrasound image; determining spatial guidance information of the ultrasound image in the target anatomical region based on a spatial prediction model corresponding to the target anatomical region. The application determines the spatial prediction model corresponding to the ultrasound image by determining the target anatomical region corresponding to the ultrasound image, and then identifies the spatial guidance information of the ultrasound image in the target anatomical region corresponding to the ultrasound image through the spatial prediction model, thereby bringing convenience to the examination of doctors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical technology, in particular to a spatial guidance method for ultrasound images and related devices. BACKGROUND

[0002] Ultrasound is widely used in the medical field due to its real-time, no radiation damage, and no side effects. Among them, two-dimensional ultrasound examination is more widely used in clinical practice due to its low cost, wide application range, and portability. However, compared with two-dimensional ultrasound image data, three-dimensional spatial data includes more information, which can help doctors shorten the ultrasound examination time. However, due to the large volume and high cost of three-dimensional ultrasound probes, it is difficult to achieve good universality, and the imaging effect of three-dimensional probes cannot reach the level of two-dimensional ultrasound probes, so three-dimensional ultrasound cannot be widely used in clinical practice.

[0003] Therefore, two-dimensional ultrasound and three-dimensional ultrasound are currently combined for use, and two-dimensional sections are selected in three-dimensional ultrasound to realize three-dimensional visualization based on two-dimensional ultrasound. However, the existing three-dimensional visualization based on two-dimensional ultrasound is realized on an in-vitro model, sensors are provided inside the in-vitro model and the ultrasound probe, spatial positioning and angle detection are performed through the sensors, and the relative position and angle of the two are determined through the detected position and angle, so that the ultrasound image and the position of the ultrasound image in the three-dimensional model can be presented simultaneously. However, the existing method needs to provide sensors for spatial positioning and angle detection inside the in-vitro model and the ultrasound probe, which cannot be applied to clinical examination.

[0004] Therefore, the existing technology needs to be improved and improved. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a spatial guidance method for ultrasound images and related devices to solve the problems of the prior art.

[0006] To solve the above technical problems, the first aspect of the embodiments of the present application provides a spatial guidance method for ultrasound images, which comprises:

[0007] Obtaining an ultrasound image to be positioned, and determining a target anatomical region corresponding to the ultrasound image;

[0008] Based on the spatial prediction model corresponding to the target anatomical region, the spatial guidance information of the ultrasound image in the target anatomical region is determined.

[0009] The spatial guidance method for ultrasound images, wherein before determining the target anatomical region corresponding to the ultrasound image, the method comprises:

[0010] detecting whether the ultrasound image is a valid ultrasound image;

[0011] when the ultrasound image is a valid ultrasound image, performing a step of determining a target anatomical region corresponding to the ultrasound image;

[0012] when the ultrasound image is a non-valid ultrasound image, discarding the ultrasound image.

[0013] The spatial guidance method for ultrasound images, wherein the step of determining a target anatomical region corresponding to the ultrasound image specifically comprises:

[0014] controlling a preset classification model to identify key anatomical structures in the ultrasound image, and determining a candidate anatomical region corresponding to the ultrasound image based on a main anatomical structure in the identified key anatomical structures;

[0015] when the number of determined candidate anatomical regions is greater than 1, selecting a target anatomical region corresponding to the ultrasound image from the determined candidate anatomical regions based on a structure weight of each anatomical structure in the identified key anatomical structures other than the main anatomical structure and a region weight corresponding to each candidate anatomical region.

[0016] when the number of determined candidate anatomical regions is equal to 1, taking the determined candidate anatomical region as the target anatomical region corresponding to the ultrasound image.

[0017] The spatial guidance method for ultrasound images, wherein the step of determining spatial guidance information of the ultrasound image in the target anatomical region based on a spatial prediction model corresponding to the target anatomical region specifically comprises:

[0018] searching for a spatial prediction model corresponding to the target anatomical region in a preset spatial prediction model database, wherein the spatial prediction model is obtained based on ultrasound images belonging to the target anatomical region;

[0019] inputting the ultrasound image into the spatial prediction model, and outputting spatial guidance information of the ultrasound image in the target anatomical region by the spatial prediction model.

[0020] The spatial guidance method for ultrasound images, wherein the spatial prediction model comprises a feature extraction module, a full connection module, and a fine-grained feature clustering module; and the step of inputting the ultrasound image into the spatial prediction model and outputting spatial guidance information of the ultrasound image in the target anatomical region by the spatial prediction model specifically comprises:

[0021] controlling the feature extraction module to extract a feature map of the ultrasound image;

[0022] The control full connection module determines a plurality of fine-grained feature vectors based on the feature map;

[0023] The control fine-grained feature clustering module generates a plurality of fine-grained feature maps based on the plurality of fine-grained feature vectors, and determines spatial guidance information of the ultrasound image in the target anatomical region based on the plurality of fine-grained feature maps.

[0024] The spatial guidance method for ultrasound images, wherein the full connection module includes a plurality of full connection layer groups; and the control full connection module determines a plurality of fine-grained feature vectors based on the feature map includes:

[0025] Each full connection layer group is controlled to generate a two-dimensional weight matrix for each channel data of the feature map, and to obtain a product of each channel data and its corresponding two-dimensional weight matrix to obtain a multi-attention feature map;

[0026] Each full connection layer group is controlled to generate a fine-grained feature vector based on the multi-attention feature map determined by itself, so as to obtain a plurality of fine-grained feature vectors.

[0027] The spatial guidance method for ultrasound images, wherein the determination of the spatial guidance information of the ultrasound image in the target anatomical region based on the plurality of fine-grained feature maps specifically includes:

[0028] The plurality of fine-grained feature maps are clustered to obtain a clustered feature map;

[0029] The spatial guidance information of the ultrasound image in the target anatomical region is determined based on the clustered feature map.

[0030] The spatial guidance method for ultrasound images, wherein the spatial prediction model includes a fusion module, and before the control fine-grained feature clustering module generates a plurality of fine-grained feature maps based on a plurality of fine-grained feature vectors, the method further includes:

[0031] The control fusion module determines a key anatomical structure class vector based on the feature map;

[0032] The control fusion module generates a class fine-grained feature vector based on the key anatomical structure class vector and a predicted key anatomical structure class vector determined when the target anatomical region is determined;

[0033] The control fusion module inputs the class fine-grained feature map into the fine-grained feature clustering module, so that the fine-grained feature clustering module generates a plurality of fine-grained feature maps based on the plurality of fine-grained feature vectors and the class fine-grained feature vector.

[0034] The spatial guidance method for the ultrasound image, wherein before the spatial guidance information of the ultrasound image in the target anatomical region is determined based on the spatial prediction model corresponding to the target anatomical region, the method further comprises:

[0035] The target anatomical region is adjusted according to the structure information of the key anatomical structure carried by the ultrasound image, and the adjusted target anatomical region is taken as the target anatomical region corresponding to the ultrasound image.

[0036] The spatial guidance method for the ultrasound image, wherein the method further comprises:

[0037] A preset target section is acquired.

[0038] Based on the spatial guidance information corresponding to the ultrasound image and the target spatial guidance information of the target section in the target anatomical region, a scanning path covering the ultrasound image and the target section is formed.

[0039] The second aspect of the embodiment of the present application provides a terminal device, which comprises a processor, a memory and a communication bus; the memory stores a computer readable program which can be executed by the processor;

[0040] The communication bus realizes the connection and communication between the processor and the memory;

[0041] The processor realizes the steps in the spatial guidance method for the ultrasound image according to any one of the above when executing the computer readable program.

[0042] Advantages: compared with the prior art, the present application provides a spatial guidance method for an ultrasound image and related devices, the method comprising acquiring an ultrasound image to be positioned, and determining a target anatomical region corresponding to the ultrasound image; based on a spatial prediction model corresponding to the target anatomical region, determining spatial guidance information of the ultrasound image in the target anatomical region. The present application determines the spatial prediction model corresponding to the ultrasound image by determining the target anatomical region corresponding to the ultrasound image, and then identifies the spatial guidance information of the ultrasound image in the target anatomical region corresponding thereto through the spatial prediction model, so that the two-dimensional ultrasound image can be presented in the three-dimensional model, which brings convenience to the doctor's examination. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor.

[0044] Figure 1 A flowchart of a method for spatial guidance of ultrasound images provided in the present application.

[0045] Figure 2 A flowchart of an example of a method for spatial guidance of ultrasound images provided in the present application.

[0046] Figure 3 A structure schematic of a positioning network model in a method for spatial guidance of ultrasound images provided in the present application.

[0047] Figure 4 A structure schematic of a positioning device for ultrasound images based on spatial information provided in the present application.

[0048] Figure 5 A structure schematic of a terminal device provided in the present application. DETAILED DESCRIPTION

[0049] The present application provides a method for spatial guidance of ultrasound images and related devices, in order to make the purpose, technical scheme and effect of the present application more clear and explicit, the present application will be further described in detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0050] Those skilled in the art can understand that, unless specifically stated otherwise, the singular form "a", "an", "said" and "the" used herein can also include the plural form. It should be further understood that the phrase "comprising" used in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0051] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0052] It should be understood that the sequence numbers and sizes of the steps in the embodiments do not mean the order of execution, and the execution order of the processes is determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0053] The inventors have found that ultrasound is widely used in the medical field due to its good real-time performance, no radiation damage, and no side effects. Among them, two-dimensional ultrasound examination is more widely used in clinical practice due to its low cost, wide application range, and convenient portability. However, compared with two-dimensional ultrasound image data, three-dimensional spatial data includes more information, which can help doctors shorten the ultrasound examination time. However, due to the characteristics of large volume and high cost of three-dimensional ultrasound probes, better universality cannot be achieved, and the imaging effect of three-dimensional probes cannot reach the effect of two-dimensional ultrasound probes, so three-dimensional ultrasound cannot be widely used in clinical practice.

[0054] Therefore, two-dimensional ultrasound and three-dimensional ultrasound are currently combined for use, and two-dimensional sections are selected in three-dimensional ultrasound to realize three-dimensional visualization based on two-dimensional ultrasound. However, the existing three-dimensional visualization based on two-dimensional ultrasound is realized on an in-vitro model, sensors are provided inside the in-vitro model and the ultrasound probe, spatial positioning and angle detection are performed through the sensors, and the relative position and angle of the two are determined through the detected position and angle, so that the ultrasound image and the position of the ultrasound image in the three-dimensional model can be presented at the same time. However, the existing method needs to provide spatial positioning and angle detection sensors inside the in-vitro model and the ultrasound probe, and cannot be applied to clinical examination.

[0055] To solve the above problems, in the embodiments of the present application, an ultrasound image to be positioned is acquired, and a target anatomical region corresponding to the ultrasound image is determined; based on a spatial prediction model corresponding to the target anatomical region, spatial guidance information of the ultrasound image in the target anatomical region is determined. The present application determines the spatial prediction model corresponding to the ultrasound image by determining the target anatomical region corresponding to the ultrasound image, and then identifies the spatial guidance information of the ultrasound image in the target anatomical region corresponding thereto through the spatial prediction model, so that the two-dimensional ultrasound image can be presented in the three-dimensional model, which brings convenience to the doctor's examination.

[0056] The application content will be further described by describing the embodiments in combination with the drawings.

[0057] The embodiments provide a spatial guidance method for an ultrasound image, as shown in Figure 1 The method comprises the following steps.

[0058] S10, acquiring an ultrasound image to be positioned, and determining a target anatomical region corresponding to the ultrasound image.

[0059] Specifically, the ultrasound image is a two-dimensional ultrasound image, wherein the ultrasound image can be a real-time ultrasound image, for example, the ultrasound image is an ultrasound image acquired in real time by an ultrasound machine or a workstation through an image data interface such as HDMI, or the positioning method provided in the embodiment is configured to the ultrasound machine to acquire a synchronous image in real time. Of course, the ultrasound image can also be a non-real-time ultrasound image, for example, the ultrasound image is an ultrasound image obtained by reading an image storage medium, etc. In addition, the ultrasound image can be an ultrasound image obtained in obstetrics, gynecology, pediatrics and other departments, or an ultrasound image of multiple parts such as the heart, breast and fetus.

[0060] The target anatomical region is an anatomical region to which the key anatomical structure carried by the ultrasound image belongs, wherein the key anatomical structure carried by the ultrasound image is a two-dimensional section view of the target anatomical region, and the target anatomical region is a three-dimensional structure of the key anatomical structure carried by the ultrasound image. Therefore, before determining the target anatomical region to which the ultrasound image belongs, it can be detected whether the ultrasound image is a valid ultrasound image, so as to avoid waste of computing power caused by invalid ultrasound image detection.

[0061] Based on this, in one implementation manner, as shown in the method provided in the embodiment, before the step of determining the target anatomical region corresponding to the ultrasound image, the method comprises: Figure 2

[0062] detecting whether the ultrasound image is a valid ultrasound image;

[0063] when the ultrasound image is a valid ultrasound image, performing the step of determining the target anatomical region corresponding to the ultrasound image;

[0064] when the ultrasound image is an invalid ultrasound image, discarding the ultrasound image.

[0065] Specifically, the valid ultrasound image refers to an ultrasound image that can carry clear image content of anatomical structures (for example, organs or structures); the invalid ultrasound image can be an ultrasound image containing only system background and / or case information. What can be accumulated is that after obtaining the ultrasound image, it can be detected whether the ultrasound image only carries system background and / or case information, if only carries system background and / or case information, there is no need to determine the target anatomical region corresponding to the ultrasound image, and the ultrasound image can be discarded directly.

[0066] ​In actual application, when the ultrasound image is acquired, it can be determined whether the ultrasound image is a valid ultrasound image, and when it is a valid ultrasound image, the spatial guidance information of the ultrasound image in the target anatomical region is determined. Or, after the ultrasound image is acquired, it is detected whether the ultrasound image is a standard section, and when the ultrasound image is a standard section, the spatial guidance information of the ultrasound image in the target anatomical region is determined. In this way, the spatial guidance information of the ultrasound image required by the doctor can be quickly determined, thereby improving the examination speed.

[0067] Based on this, in a typical implementation, the preset section to be checked can be set in advance. When the ultrasound image is acquired, it is detected whether the ultrasound image is the preset section (i.e., the preset two-dimensional ultrasound section). When it is the preset section to be checked, it is determined that the ultrasound image is a target ultrasound image. At this time, the step of determining the target anatomical region corresponding to the ultrasound image can be performed. Conversely, when it is not the preset section to be checked, it is determined that the ultrasound image is a non-target ultrasound image. At this time, the ultrasound image can be discarded and the step of acquiring the ultrasound image can be continued. In this way, the preset section is used to screen the ultrasound image, which can reduce the number of executions of the positioning method provided in the embodiment, thereby reducing the working pressure of the electronic device configured with the positioning method provided in the embodiment and improving the calculation speed and efficiency.

[0068] In an implementation of the embodiment, the process of detecting whether the ultrasound image is a valid ultrasound image can be implemented by a traditional image processing method or a machine learning processing method. The traditional image processing method can remove invalid background or interference information by methods such as image erosion, threshold segmentation, and gray scale calculation, thereby determining whether the ultrasound image is a valid ultrasound image. The machine learning processing method can determine the image category of the acquired ultrasound image by using methods including vector machines, random forests, and deep neural networks to determine whether it is a valid ultrasound image. Of course, in actual application, other methods can also be used for detection, which will not be described here.

[0069] Different anatomical regions can include the same key anatomical structure, for example, the heart exists in both prenatal fetal ultrasound and adult ultrasound. When the heart section is detected, the adult heart region and the prenatal fetal heart region are obtained, so that the target anatomical region of the ultrasound image cannot be accurately determined. Therefore, in an implementation, the determination of the target anatomical region corresponding to the ultrasound image specifically includes:

[0070] Controlling a preset classification model to identify the key anatomical structure in the ultrasound image, and determining the candidate anatomical region corresponding to the ultrasound image based on the main anatomical structure in the identified key anatomical structure.

[0071] When the number of determined candidate anatomical regions is greater than 1, a target anatomical region corresponding to the ultrasound image is determined from the candidate anatomical regions based on the structure weights of the identified anatomical structures other than the main anatomical structure and the region weights of the candidate anatomical regions.

[0072] Specifically, the classification model is used to identify the key anatomical structure in the ultrasound image, and determine the target anatomical region to which the ultrasound image belongs based on the identified key anatomical structure. The classification model can adopt a traditional image processing method or a machine learning processing method. The traditional image processing method can determine the image content of the ultrasound image according to known prior knowledge through edge extraction, threshold segmentation, frequency domain transformation and other analysis methods, so as to determine the key anatomical structure carried by the ultrasound image, and further determine the target anatomical region corresponding to the ultrasound image. The machine learning processing method can use deep neural networks such as VGG, Inception and Resnet to classify and predict the ultrasound image, so as to obtain the target anatomical region corresponding to the ultrasound image. In a typical implementation, the classification model can use a multi-branch detection classification model (such as MaskRcnn, Yolact, etc. target detection network), detect the key anatomical structure through the detection branch, determine whether the ultrasound image is a valid ultrasound image based on the detected key anatomical result, and then predict the target anatomical region to which the ultrasound image belongs through the classification branch, so as to realize accurate classification of the ultrasound image. In this way, the valid ultrasound image detection and the target anatomical region detection are combined, so that the target anatomical region where the ultrasound image is located does not need to be separately detected, thereby reducing the operation steps and improving the positioning efficiency.

[0073] The ultrasound image can include several key anatomical structures, and the main key anatomical structure can be the key anatomical structure with the largest image proportion in the several key anatomical structures. That is, after obtaining the several key anatomical structures, the image proportion of each joint anatomical structure in the ultrasound image can be determined, and then the main key anatomical structure can be selected from the several key anatomical structures based on the image proportion. Of course, other methods can also be used in actual applications, for example, each key anatomical structure in the several key anatomical structures carries a weight factor, and the main key anatomical structure can be selected from the several key anatomical structures based on the weight factor, such as selecting the key anatomical structure with the largest weight factor as the main key anatomical structure, etc. In addition, when two main key anatomical structures are selected from the several key anatomical structures, a previous image frame of the ultrasound image can be obtained, and the main key anatomical structure corresponding to the ultrasound image can be determined based on the main key anatomical structure corresponding to the previous image frame, wherein the main key anatomical structure corresponding to the previous image frame is the same as the main key anatomical structure corresponding to the ultrasound image.

[0074] After the main key anatomical structure is determined, a candidate anatomical region corresponding to the ultrasound image is determined according to the main key anatomical structure. The candidate anatomical region can be one or multiple. When the number of the candidate anatomical regions is one, the candidate anatomical region is directly taken as the target anatomical region corresponding to the ultrasound image. When the number of the candidate anatomical regions is greater than one, the structure weight of each anatomical structure except the main anatomical structure in the identified key anatomical structures and the region weight of each candidate anatomical region are obtained, wherein the structure weight and the region weight can be determined by the classification model. That is, when the classification model determines each key anatomical structure in the ultrasound image, the structure weight corresponding to each key anatomical structure is determined. When the candidate anatomical region is determined based on the main anatomical structure, the region weight of each candidate anatomical region is also determined.

[0075] After the structure weight and the region weight are obtained, for each candidate anatomical region, an anatomical structure belonging to the candidate anatomical region is selected from the anatomical structures except the main anatomical structure, and the region weight of the candidate region is updated based on the structure weight of the selected anatomical structure and the region weight of the candidate region. Then, the candidate anatomical region with the maximum updated region weight is taken as the target anatomical region. The updating of the region weight of the candidate region based on the structure weight of the selected anatomical structure and the region weight of the candidate region can take the sum of the structure weight of the selected anatomical structure and the region weight of the candidate region as the updated region weight of the candidate region, or take the product of the structure weight of the selected anatomical structure and the region weight of the candidate region as the updated region weight of the candidate region, and so on.

[0076] For example, the main anatomical structure corresponding to the ultrasound image is the heart structure, and the anatomical structures except the main anatomical structure include amniotic fluid. The candidate anatomical region determined based on the heart structure includes adult heart and prenatal fetal heart. The region weight of the adult heart is 0.9, the region weight of the prenatal fetal heart is 0.8, and the structure weight of the amniotic fluid is 1.2. Then, the region weight of the prenatal fetal heart can be adjusted to 1.2*0.8=0.96, and the region weight of the adult heart remains 0.9. Thus, the target anatomical region corresponding to the ultrasound image can be determined as the prenatal fetal heart.

[0077] S20, based on the spatial prediction model corresponding to the target anatomical region, determining the spatial guidance information of the ultrasound image in the target anatomical region.

[0078] Specifically, the spatial prediction model is used to determine the spatial guide information of the ultrasound image in the target anatomical region, and each ultrasound image has a unique spatial position in the target anatomical region. That is, the target anatomical region is a three-dimensional anatomical structure, the ultrasound image is a two-dimensional section of the three-dimensional anatomical structure, the spatial guide information is the position information of the two-dimensional section in the three-dimensional anatomical structure, and the two-dimensional section can be uniquely determined in the three-dimensional anatomical structure based on the spatial guide information. For example, the target anatomical region is a prenatal fetus, and the ultrasound image is a fetal heart section. Then, the spatial guide information is the position of the front of the fetal heart in the three-dimensional fetal model, and the fetal heart section can be located in the fetal model based on the spatial guide information, so that the three-dimensional visualization of the fetal heart section is displayed.

[0079] The spatial guide information can be set as a position vector, for example, the spatial guide information can be represented as (p1, p2, p3, p4, …, pn), wherein the position vector can include spatial position and offset information, etc. After obtaining the spatial guide information, the section equation of the ultrasound image in the target anatomical region can be determined based on the position vector, so that the ultrasound image in the target anatomical region can be quickly located.

[0080] In addition, the development degree of the target anatomical structure of the target anatomical region may be different at different periods, for example, the size of the fetus at different periods, the hip joint at different development degrees. Therefore, before determining the spatial guide information, the target anatomical region can be adjusted based on the measured value of the biological parameter of the key anatomical structure in the ultrasound image, which can improve the accuracy of the three-dimensional visualization of the ultrasound image.

[0081] Therefore, before determining the spatial guide information of the ultrasound image in the target anatomical region based on the spatial prediction model corresponding to the target anatomical region, the method further comprises:

[0082] Adjusting the target anatomical region according to the structure information of the key anatomical structure carried by the ultrasound image, and taking the adjusted target anatomical region as the target anatomical region corresponding to the ultrasound image.

[0083] Specifically, the structure information of the key anatomical structure can include the measured value of the biological parameter of the key anatomical structure in the ultrasound image, for example, the structure size of the key anatomical structure, etc., and can also include the direction information of the key anatomical structure, etc. The adjustment of the target anatomical region means adjusting the three-dimensional anatomical model template corresponding to the target anatomical region, so that the three-dimensional anatomical model template matches the ultrasound image, wherein the adjustment can include stretching, scaling of each direction of the three-dimensional anatomical model template, and / or adjusting part of the anatomical model in the three-dimensional anatomical model template, etc.

[0084] In an implementation form of the embodiment, the determining the spatial guidance information of the ultrasound image in the target anatomical region based on the spatial prediction model corresponding to the target anatomical region specifically comprises:

[0085] finding the spatial prediction model corresponding to the target anatomical region in a preset spatial prediction model database, wherein the spatial prediction model is obtained based on ultrasound images belonging to the target anatomical region;

[0086] inputting the ultrasound image into the spatial prediction model, and outputting the spatial guidance information of the ultrasound image in the target anatomical region by the spatial prediction model.

[0087] Specifically, the spatial prediction model database stores a plurality of spatial prediction models, the plurality of spatial prediction models have the same model structure, and the spatial prediction models differ in training data sets used for training the spatial prediction models. The training data set corresponding to each spatial prediction model corresponds to an anatomical structure, and the training data sets correspond to different anatomical structures, so as to improve the positioning accuracy of the two-dimensional ultrasound section of each spatial prediction model for the corresponding anatomical structure. That is, after the ultrasound image is obtained, a spatial prediction model trained by using training ultrasound images belonging to the target anatomical region corresponding to the ultrasound image can be selected from the spatial prediction model database, so as to improve the accuracy of the determined spatial guidance information of the ultrasound image.

[0088] In an implementation form, the spatial prediction model comprises a feature extraction module, a full connection module, and a fine-grained feature clustering module; the inputting the ultrasound image into the spatial prediction model and outputting the spatial guidance information of the ultrasound image in the target anatomical region by the spatial prediction model specifically comprises:

[0089] controlling the feature extraction module to extract a feature map of the ultrasound image;

[0090] controlling the full connection module to determine a plurality of fine-grained feature vectors based on the feature map;

[0091] controlling the fine-grained feature clustering module to generate a plurality of fine-grained feature maps based on the plurality of fine-grained feature vectors, and determining the spatial guidance information of the ultrasound image in the target anatomical region based on the plurality of fine-grained feature maps.

[0092] Specifically, as Figure 3As shown, the feature extraction module is connected with the full connection module, and the full connection module is connected with the fine-grained feature clustering module. The input of the feature extraction module is an ultrasound image, and the output is a feature map corresponding to the ultrasound image. The input of the full connection module is the feature map, and the output is a plurality of fine-grained feature vectors. The input of the fine-grained feature clustering module is the plurality of fine-grained feature vectors, and the output is spatial guidance information. The number of the plurality of fine-grained feature vector can be the same as the number of the plurality of fine-grained feature map, or can not be the same as the number of the plurality of fine-grained feature map. In addition, the feature extraction module can use a lightweight learning network such as MobileNet, ShuffleNet, SqueezeNet or GhostNet to extract image features in the ultrasound image to obtain the feature map.

[0093] In one implementation, the full connection module includes a plurality of full connection layer groups, and each full connection layer group corresponds to a fine-grained feature vector. Based on this, the control full connection module determines a plurality of fine-grained feature vectors based on the feature map, which includes:

[0094] respectively controlling each full connection layer group to generate a two-dimensional weight matrix for each channel data of the feature map, and obtaining the product of each channel data and its corresponding two-dimensional weight matrix to obtain a multi-attention feature map;

[0095] respectively controlling each full connection layer group to generate a fine-grained feature vector based on the multi-attention feature map determined by itself, to obtain a plurality of fine-grained feature vectors.

[0096] Specifically, the input of the full connection layer group is the feature map, and the feature map includes a plurality of channel data. The full connection layer generates a two-dimensional weight matrix for each channel data in the plurality of channel data. The two-dimensional weight matrix corresponding to each channel data is different from each other. The matrix dimension of the two-dimensional weight matrix is the same as the data dimension of the channel data, that is, the number of matrix elements included in each row of the two-dimensional weight matrix is equal to the number of pixels included in the width direction of the channel data, and the number of matrix elements included in each column of the two-dimensional weight matrix is equal to the number of pixels included in the height direction of the channel data. After obtaining the two-dimensional weight matrix, the two-dimensional weight matrix is multiplied by the channel data to obtain a multi-attention feature map, and then the multi-attention feature map is full connected to obtain a fine-grained feature vector. In addition, each full connection layer group can include a plurality of cascaded full connection layers, for example, each full connection layer group includes three cascaded full connection layers.

[0097] In an implementation form of the embodiment, the clustered feature map is obtained by clustering a plurality of fine-grained feature maps. Based on this, the determining the spatial guidance information of the ultrasound image in the target anatomical region based on the plurality of fine-grained feature maps specifically includes: clustering the plurality of fine-grained feature maps to obtain a clustered feature map; and determining the spatial guidance information of the ultrasound image in the target anatomical region based on the clustered feature map. Wherein, the plurality of fine-grained feature maps are formed based on a plurality of fine-grained feature vectors, and each fine-grained feature map can be formed by splicing a plurality of fine-grained feature vectors according to different weights.

[0098] In an implementation form, the spatial prediction model includes a fusion module, and before the controlling the fine-grained feature clustering module to generate a plurality of fine-grained feature maps based on a plurality of fine-grained feature vectors, the method further includes:

[0099] controlling the fusion module to determine a key anatomical structure category vector based on the feature map;

[0100] controlling the fusion module to generate a category fine-grained feature vector based on the key anatomical structure category vector and a predicted key anatomical structure category vector determined when the target anatomical region is determined;

[0101] controlling the fusion module to input the category fine-grained feature map into the fine-grained feature clustering module, so that the fine-grained feature clustering module generates a plurality of fine-grained feature maps based on the plurality of fine-grained feature vectors and the category fine-grained feature vector.

[0102] Specifically, the key anatomical structure category vector is determined by the fusion module based on the feature map extracted by the feature extraction module, and the predicted key anatomical structure category vector is determined when the target anatomical region is determined, for example, by a classification model for determining the target anatomical region. After obtaining the key anatomical structure category vector and the predicted key anatomical structure category vector, the key anatomical structure category vector and the predicted key anatomical structure category vector are fused to obtain a category fine-grained feature vector, wherein the key anatomical structure category vector and the predicted key anatomical structure category vector can be added, or the key anatomical structure category vector and the predicted key anatomical structure category vector can be weighted, etc.

[0103] After the obtained category fine-grained feature vector, the category fine-grained feature vector is taken as a fine-grained feature vector, and a plurality of fine-grained feature vectors are determined by the full connection module as input items of the clustering module, so that the clustering module generates a plurality of fine-grained feature maps based on the fine-grained feature vectors and the category fine-grained feature vector. In this embodiment, the category fine-grained feature vector is taken as a fine-grained feature vector, which can increase the image feature information, thereby improving the accuracy of the spatial guidance information.

[0104] In an implementation manner, the method further includes:

[0105] obtaining a preset target section;

[0106] forming a scanning path covering the ultrasound image and the target section based on the spatial guidance information corresponding to the ultrasound image and the target spatial guidance information of the target section in the target anatomical region.

[0107] Specifically, the target section is preset, and the target spatial guidance information of each target section in the target anatomical region is known. Therefore, after obtaining the spatial guidance information corresponding to the ultrasound image, a scanning path can be formed between the ultrasound image and the target section, and when the ultrasound probe is controlled to move according to the scanning path, the ultrasound image corresponding to the target section can be collected. In this way, the doctor can quickly scan the ultrasound image corresponding to the target section. In addition, in actual application, the target section can be multiple, and each target section can carry a section priority. When the scanning path is generated, the scanning order of each target section can be determined based on the section priority, and then the scanning path can be formed based on the scanning order, the spatial guidance information corresponding to the ultrasound image and the target spatial guidance information. When scanning according to the scanning path, the ultrasound image corresponding to each target section can be scanned in turn according to the scanning order corresponding to each target section. In addition, after obtaining the scanning path, the scanning path can be automatically updated according to the target section after switching when the target section is switched.

[0108] In an implementation manner of the embodiment, after obtaining the scanning path, the scanning path, the ultrasound image and the target section in the target anatomical region can be displayed to realize three-dimensional visualization of the two-dimensional section. Of course, in actual application, when the scanning path is displayed in the three-dimensional anatomical structure, the scanning path can be rotated, translated and indicated, so as to better indicate the scanning process of the doctor and improve the scanning efficiency.

[0109] In summary, the embodiment provides a spatial guidance method for an ultrasound image and a related device. The method includes obtaining an ultrasound image to be positioned, and determining a target anatomical region corresponding to the ultrasound image. Based on a spatial prediction model corresponding to the target anatomical region, spatial guidance information of the ultrasound image in the target anatomical region is determined. The application determines the spatial prediction model corresponding to the ultrasound image by determining the target anatomical region corresponding to the ultrasound image, and then identifies the spatial guidance information of the ultrasound image in the target anatomical region corresponding to the ultrasound image through the spatial prediction model, so that the two-dimensional ultrasound image can be presented in the three-dimensional model, which brings convenience to the doctor's examination.

[0110] Based on the above-mentioned spatial guidance method for an ultrasound image, the embodiment provides a positioning device for an ultrasound image based on spatial information, as shown in Figure 4 The device includes:

[0111] The obtaining module 100 is configured to obtain an ultrasound image to be positioned, and determine a target anatomical region corresponding to the ultrasound image.

[0112] The determining module 200 is configured to determine spatial guidance information of the ultrasound image in the target anatomical region based on a spatial prediction model corresponding to the target anatomical region.

[0113] Based on the above-mentioned spatial guidance method for an ultrasound image, the embodiment provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the spatial guidance method for an ultrasound image as described in the above embodiment.

[0114] Based on the above-mentioned spatial guidance method for an ultrasound image, the application further provides a terminal device, as shown in Figure 5 The terminal device includes at least one processor 20, a display screen 21, a memory 22, and a communications interface 23, and a bus 24. The processor 20, the display screen 21, the memory 22, and the communications interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guidance interface in an initial setting mode. The communications interface 23 can transmit information. The processor 20 can call logical instructions in the memory 22 to execute the method in the above embodiment.

[0115] In addition, the logical instructions in the above-mentioned memory 22 can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium.

[0116] The memory 22, as a computer readable storage medium, can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the method in the embodiments of the present disclosure. The processor 20 executes the functions of the application and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implements the method in the above embodiments.

[0117] The memory 22 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory. For example, a variety of media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. can also be a transitory storage medium.

[0118] In addition, the specific process of the memory medium and the plurality of instructions in the terminal device loaded and executed by the processor has been described in detail in the above method, and will not be repeated here.

[0119] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A spatial guidance method for ultrasound images, characterized in that, The method includes: Acquire the ultrasound image to be located and determine the target anatomical region corresponding to the ultrasound image; Based on the spatial prediction model corresponding to the target anatomical region, the spatial guidance information of the ultrasound image in the target anatomical region is determined; The spatial prediction model includes a feature extraction module, a fully connected module, and a fine-grained feature clustering module. The fully connected module includes several fully connected layer groups. The process of determining the spatial guidance information of the ultrasound image in the target anatomical region specifically includes: The feature extraction module is controlled to extract feature maps from the ultrasound image; Each fully connected layer group is controlled to generate a two-dimensional weight matrix for each channel data of the feature map, and the product of each channel data and its corresponding two-dimensional weight matrix is ​​obtained to obtain a multi-attention feature map. Each fully connected layer group is controlled to generate fine-grained feature vectors based on its own determined multi-attention feature map, so as to obtain several fine-grained feature vectors; The fine-grained feature clustering module is controlled to generate several fine-grained feature maps based on several fine-grained feature vectors, and to determine the spatial guidance information of the ultrasound image in the target anatomical region based on the several fine-grained feature maps.

2. The spatial guidance method for ultrasound images according to claim 1, characterized in that, Determining the target anatomical region corresponding to the ultrasound image specifically includes: The system controls a preset classification model to identify key anatomical structures in the ultrasound image and determines the candidate anatomical region corresponding to the ultrasound image based on the main anatomical structure in the identified key anatomical structures. When the number of identified candidate anatomical regions is greater than 1, the target anatomical region corresponding to the ultrasound image is selected from the identified candidate anatomical regions based on the structural weights of each anatomical structure other than the main anatomical structure in the identified key anatomical structures and the region weights corresponding to each candidate anatomical region. When the number of determined candidate anatomical regions is equal to 1, the determined candidate anatomical region is taken as the target anatomical region corresponding to the ultrasound image.

3. The spatial guidance method for ultrasound images according to claim 1, characterized in that, The determination of spatial guidance information of the ultrasound image in the target anatomical region based on the spatial prediction model corresponding to the target anatomical region specifically includes: The spatial prediction model corresponding to the target anatomical region is searched in a preset spatial prediction model database, wherein the spatial prediction model is obtained by training on ultrasound images belonging to the target anatomical region. The ultrasound image is input into the spatial prediction model, and the spatial prediction model outputs spatial guidance information of the ultrasound image in the target anatomical region.

4. The spatial guidance method for ultrasound images according to claim 1, characterized in that, The determination of spatial guidance information of the ultrasound image in the target anatomical region based on the aforementioned fine-grained feature maps specifically includes: Clustering is performed on the aforementioned fine-grained feature maps to obtain clustered feature maps; Based on the clustering feature map, spatial guidance information of the ultrasound image in the target anatomical region is determined.

5. The spatial guidance method for ultrasound images according to claim 1, characterized in that, The spatial prediction model includes a fusion module. Before the fine-grained feature clustering module generates several fine-grained feature maps based on several fine-grained feature vectors, the method further includes: The fusion module is controlled to determine the key anatomical structure category vectors based on the feature map; The fusion module is controlled to generate a fine-grained feature vector based on the key anatomical structure category vector and the predicted key anatomical structure category vector determined when determining the target anatomical region. The fusion module is controlled to input the category fine-grained feature vector into the fine-grained feature clustering module, so that the fine-grained feature clustering module generates several fine-grained feature maps based on the several fine-grained feature vectors and the category fine-grained feature vectors.

6. The spatial guidance method for ultrasound images according to any one of claims 1-5, characterized in that, Before determining the spatial guidance information of the ultrasound image in the target anatomical region based on the spatial prediction model corresponding to the target anatomical region, the method further includes: The target anatomical region is adjusted based on the structural information of the key anatomical structures carried by the ultrasound image, and the adjusted target anatomical region is taken as the target anatomical region corresponding to the ultrasound image.

7. The spatial guidance method for ultrasound images according to any one of claims 1-5, characterized in that, The method further includes: Obtain the preset target section; Based on the spatial guidance information corresponding to the ultrasound image and the target spatial guidance information of the target section within the target anatomical region, a scanning path covering the ultrasound image and the target section is formed.

8. A terminal device, characterized in that, include: Processor, memory, and communication bus; The memory stores a computer-readable program that can be executed by the processor; The communication bus enables communication between the processor and the memory; When the processor executes the computer-readable program, it implements the steps of the spatial guidance method for ultrasound images as described in any one of claims 1-7.

Images