Ultrasonic image recognition method, device and equipment and readable storage medium

Abstract

The invention discloses an ultrasonic image recognition method, device and equipment and a readable storage medium, and the method comprises the steps: carrying out the feature extraction of an ultrasonic scanning section image, and obtaining the feature information corresponding to an overall structure and an internal structure; the feature information comprises structure position features and structure content features; performing association relationship analysis on the position features and the structure content features of the different structures to obtain association information; fusing the associated information into the feature information to correct the feature information through feature fusion to obtain corrected feature information; and outputting structure identification information respectively corresponding to the overall structure and the internal structure by utilizing the corrected feature information. According to the method, association relationship analysis is carried out on positions and contents of different structures, association information is fused into feature information, and the feature information of the structures is corrected based on the feature association relationship between the different structures, so that the detection accuracy of each structure is improved.

Description

Technical Field

[0001] This application relates to the field of ultrasound technology, and in particular to an ultrasound image recognition method, apparatus, device, and readable storage medium. Background Technology

[0002] Ultrasound examinations often require outputting many slices. For example, obstetric ultrasound examinations include three levels of screening, covering multiple parts of the fetus. According to prenatal ultrasound screening guidelines, these ultrasound scans had a total of 75 standard slices, and there were hundreds of different key structures within each slice.

[0003] To facilitate doctors' examinations, current intelligent ultrasound scanning can make overall judgments on standard sections. However, when it comes to identifying small structures, it is easy to produce a large number of false positives for small structures.

[0004] In summary, how to effectively solve the problem of accuracy in ultrasonic scanning cross-sectional image recognition is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide an ultrasound image recognition method, device, equipment, and readable storage medium. By analyzing the correlation between the position and content of different structures, the correlation information is integrated into the feature information, thereby correcting the feature information of the structure based on the feature correlation between different structures, and thus improving the accuracy of detecting each structure.

[0006] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0007] An ultrasound image recognition method, comprising:

[0008] Feature extraction is performed on the ultrasonic scanning section image to obtain feature information corresponding to the overall structure and internal structure, respectively; the feature information includes structural location features and structural content features.

[0009] The correlation information is obtained by analyzing the relationship between the structural position features and structural content features of different structures.

[0010] The associated information is fused into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information;

[0011] Using the corrective feature information, output the structural identification information corresponding to the overall structure and the intrinsic structure respectively.

[0012] For example, correlation analysis is performed on the structural position features and structural content features between different structures to obtain correlation information, including:

[0013] Based on the structural location features and structural content features, correlation analysis is performed on pairs of different structures to obtain correlation information between pairs of different structures; wherein, different structures correspond to the overall structure and the internal structure, or, different structures correspond to different organizational structures within the internal structure.

[0014] For example, based on the structural location features and the structural content features, correlation analysis is performed on pairs of different structures to obtain correlation information between pairs of different structures, including:

[0015] Correlation analysis is performed on the structural positional features of two different structures to obtain the feature relationships between the two different structures;

[0016] By performing correlation analysis on the structural content features of two different structures, the geometric relationships between the two different structures are obtained;

[0017] By integrating the feature relationships and the geometric relationships, we can obtain the association information of two different structures.

[0018] For example, the characteristic relationship indicates whether there is an association between structures; the geometric relationship indicates the relative position and size relationship between structures.

[0019] Accordingly, the associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including:

[0020] Using the aforementioned feature relationships, the occurrence rationality of the feature information is detected, and a rationality detection result is obtained.

[0021] Using the geometric relationship, geometric reliability detection is performed on the feature information to obtain reliability detection results;

[0022] By combining the rationality test results and the reliability test results, the feature information is corrected to obtain the corrected feature information.

[0023] For example, by combining the rationality detection result and the reliability detection result, the feature information is corrected to obtain the corrected feature information, including:

[0024] If the rationality detection results determine that there is a target feature that should not appear, then the feature related to the target feature is deleted from the feature information;

[0025] If the reliability test results determine that there are errors in the relative position and / or size relationship in the feature information, then the relative position and / or size relationship in the feature information shall be adjusted.

[0026] The feature information after feature deletion and content adjustment is determined as the corrected feature information.

[0027] For example, integrating the feature relationships and the geometric relationships yields association information between pairs of different structures, including:

[0028] The feature relationship and the geometric relationship are quantized respectively to obtain the quantization results;

[0029] Based on the quantification results, the correlation weights between each structure and other structures are calculated to obtain the correlation information between different structures.

[0030] Accordingly, the associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including:

[0031] The corrected feature information for each structure is obtained by weighted summation of all associated weights for each structure.

[0032] For example, correlation analysis is performed on the structural position features and structural content features between different structures to obtain correlation information, including:

[0033] A relationship matrix is ​​constructed using the structural location features and the structural content features;

[0034] Using the aforementioned relationship matrix, calculate the correlation parameter values ​​between every two structures;

[0035] The correlation parameter values ​​between every two structures are determined as the correlation information.

[0036] For example, the ultrasound image recognition method is implemented using a deep learning model, which includes fully connected layers and / or convolutional layers;

[0037] The associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including:

[0038] The association information and the feature information are input into a fully connected layer for fully connected processing, and the output of the fully connected layer is determined as the corrected feature information.

[0039] or,

[0040] The associated information and the feature information are input into a convolutional layer for convolution calculation, and the output of the convolutional layer is determined as the corrected feature information.

[0041] An ultrasound image recognition device, comprising:

[0042] The feature extraction module is used to extract features from the ultrasound scan section image to obtain feature information corresponding to the overall structure and the internal structure, respectively; the feature information includes structural location features and structural content features;

[0043] The interactive association module is used to analyze the association relationship between the structural position features and structural content features of different structures to obtain association information;

[0044] A feature correction module is used to fuse the associated information into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information;

[0045] The structure detection module is used to output the structural identification information corresponding to the overall structure and the internal structure respectively using the correction feature information.

[0046] An electronic device, comprising:

[0047] Memory, used to store computer programs;

[0048] A processor is used to implement the steps of the ultrasound image recognition method described above when executing the computer program.

[0049] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the ultrasound image recognition method described above.

[0050] Using the method provided in the embodiments of this application, feature extraction is performed on the ultrasonic scanning section image to obtain feature information corresponding to the overall structure and the internal structure, respectively. The feature information includes structural position features and structural content features. The correlation relationship between the structural position features and the structural content features of different structures is analyzed to obtain correlation information. The correlation information is fused into the feature information to correct the feature information through feature fusion to obtain corrected feature information. Using the corrected feature information, structural identification information corresponding to the overall structure and the internal structure is output.

[0051] In this application, feature extraction is first performed on the ultrasonic scanning section image to obtain structural location features and structural content features corresponding to the overall structure and internal structure, respectively. Correlation analysis is then performed on the structural location features and structural content features to obtain correlation information. This correlation information is then fused into the feature information, thereby correcting the feature information through feature fusion to obtain corrected feature information. In other words, the correlation information obtained through correlation analysis can fuse feature information from different structures, correcting the structural feature information and resulting in more accurate output of structural identification information corresponding to the overall structure and internal structure based on the corrected feature information.

[0052] The technical effect of this application is that by analyzing the correlation between the position and content of different structures, the correlation information is integrated into the feature information, and the feature information of the structure is corrected based on the feature correlation between different structures, thereby improving the accuracy of detecting each structure.

[0053] Accordingly, embodiments of this application also provide an ultrasonic image recognition device, equipment, and readable storage medium corresponding to the above-described ultrasonic image recognition method, which have the above-described technical effects, and will not be repeated here. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the implementation of an ultrasound image recognition method in this application.

[0056] Figure 2 This is a flowchart illustrating a specific implementation of an ultrasound image recognition method in this application.

[0057] Figure 3 This is a schematic diagram illustrating the model structure and working principle in an embodiment of this application;

[0058] Figure 4 This is a schematic diagram of the structure of an ultrasonic image recognition device according to an embodiment of this application;

[0059] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0060] Figure 6 This is a schematic diagram of the specific structure of an electronic device in an embodiment of this application. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] The ultrasound image recognition method provided in this application can be implemented using an electronic device. This electronic device can be a server or a terminal device. The terminal device can be a personal computer, an ultrasound device, etc. If the electronic device is not an ultrasound device, it can acquire ultrasound scan section images from the ultrasound device via wired or wireless means.

[0063] Please refer to Figure 1 , Figure 1 This is a flowchart of an ultrasound image recognition method according to an embodiment of this application. The method includes the following steps:

[0064] S101. Extract features from the ultrasound scan section image to obtain feature information corresponding to the overall structure and internal structure, respectively.

[0065] The feature information includes structural location features and structural content features.

[0066] Ultrasonic scanning section images can be acquired in real time by ultrasonic scanning equipment, or they can be pre-scanned and stored by ultrasonic scanning equipment and retrieved from the storage device, or they can be obtained by downloading or receiving from ultrasonic scanning equipment.

[0067] The ultrasound scan cross-sectional image can be a cross-sectional image obtained from scanning various body parts. Furthermore, the ultrasound scan cross-sectional image can be an image obtained from an obstetric ultrasound examination corresponding to a relatively large number of internal structures. For example, it can correspond to any of the primary, secondary, and tertiary screenings in obstetric ultrasound examinations. The ultrasound scan cross-sectional image can be a standard cross-sectional image or a non-standard cross-sectional image.

[0068] In this embodiment, after acquiring the ultrasound scan section image, feature extraction can be performed on the ultrasound scan section image. This feature extraction can be done through global feature extraction, local feature extraction, or other methods, thereby obtaining feature information corresponding to the overall structure and the internal structure, respectively. The overall structure can be the section corresponding to the entire current ultrasound scan section image, such as a cross-section of the upper abdomen, or the overall structure corresponding to a body organ or part on the section image. The internal structure can be a small structure located within the overall structure, such as the femoral head or umbilical vein. It should be noted that the number of both the overall structure and the internal structure can be at least one. Taking a fetal scan section as an example, a certain ultrasound scan section image can correspond to two overall structures: the upper abdominal structure and the hand structure. The upper abdominal structure can also include internal structures such as the femoral head, umbilical vein, spine, and stomach bubble. In this embodiment, the overall structure is also referred to as the large structure, and the internal structure is referred to as the small structure.

[0069] Features related to location are called structural location features; features of the structure itself are called structural content features. For example, location features refer to the position of the structure in the ultrasound scan cross-sectional image, while structural content features refer to the pixel features, shape features, name, number, etc. of the structure. Among these, pixel features, shape features, etc., can be collectively referred to as appearance features or graphic features, and the name can be a textual representation such as the spine.

[0070] In practical applications, deep learning models can be used to extract features from ultrasound scan cross-sectional images. It is sufficient that the deep learning model can extract feature information corresponding to both the overall structure and the intrinsic structure. The specific training method, internal structure, and processing logic of the deep learning model are not limited in this embodiment. The same model can output feature information corresponding to both the overall structure and the intrinsic structure simultaneously, or one model can output feature information corresponding to the overall structure while another model outputs feature information corresponding to the intrinsic structure. Even feature information corresponding to different intrinsic structures can be output by different models. It is understood that the more models there are, the more targeted the model becomes, and the higher the accuracy of the output feature information.

[0071] S102. Analyze the correlation between the structural positional features and structural content features of different structures to obtain correlation information.

[0072] Because the human body has certain regularities in its structure, there are some relationships between its structures. For example, small structures related to the lower limbs will not appear in a cross-section of the upper abdomen; the positions of small structures within larger structures are also relatively fixed; and the positions and occurrences of different small structures are also relatively fixed. In other words, there are relationships between structures.

[0073] In this embodiment, by interactively processing structural location features and structural content features, this interactive processing analyzes the correlation between different structures, thereby obtaining correlation information between different structures. This correlation information includes, but is not limited to, positional relationships and geometric relationships.

[0074] In one specific embodiment of this application, the correlation analysis of the structural position features and structural content features between different structures is performed to obtain correlation information, including: performing correlation analysis on the structural position features and structural content features of two different structures to obtain correlation information between two different structures; wherein, different structures correspond to the overall structure and the internal structure, or, different structures correspond to different organizational structures in the internal structure.

[0075] In this embodiment, interactive processing can specifically involve analyzing the correlation between the structural positional features and structural content features of every two different structures to obtain the correlation information between the two different structures. The objects of analysis can be the overall structure and its internal structure, or different organizational structures within the internal structure. That is, correlation analysis can be performed between large and small structures, or between two different small structures. Furthermore, the correlation analysis is not limited to a set of different structures, but can also be between every two structures within a large or small structure. Additionally, correlation analysis can be performed on different overall structures, or on the internal structures under different overall structures. In short, correlation analysis can be performed on any two structures identified from the ultrasonic scanning section image.

[0076] Specifically, correlation analysis is performed on pairs of different structures based on their structural location features and structural content features to obtain correlation information between the pairs of different structures, including:

[0077] Correlation analysis is performed on the structural positional features of two different structures to obtain the characteristic relationships between the two different structures.

[0078] By performing correlation analysis on the structural content features of two different structures, the geometric relationships between the two different structures can be obtained;

[0079] By integrating feature relationships and geometric relationships, we can obtain the correlation information between pairs of different structures.

[0080] In other words, in this embodiment, when performing correlation analysis on two different structures, correlation analysis can be performed on their structural position features separately to obtain the feature relationship between the two different structures; correlation analysis can be performed on their structural content features to obtain the geometric relationship between the two different structures. Finally, by integrating the feature relationship and the geometric relationship, the correlation information between the two different structures can be obtained.

[0081] Among them, characteristic relationships indicate whether there is a connection between structures; geometric relationships indicate the relative position and size relationship between structures. For example, characteristic relationships can indicate whether two different structures should appear simultaneously in an ultrasound scan image. For instance, the stomach bubble, umbilical vein, and spine usually appear simultaneously in the standard upper abdominal structure, while the femoral head and other structures such as the umbilical vein usually do not appear simultaneously. Geometric relationships can indicate the specific position and size relationship of two different structures in an ultrasound scan image. For instance, the relationship between the stomach bubble and the upper abdominal structure is that the stomach bubble is surrounded by the overall structure, and the stomach bubble, spine, and umbilical vein usually form a triangular relationship in position.

[0082] The initially obtained feature relationships and geometric relationships can be qualitative information. These relationships can then be quantified separately to facilitate subsequent feature fusion. For example: 1) For feature relationships, 1 can represent that two structures should appear simultaneously, and 0 can represent that they should not appear simultaneously. For geometric relationships, the relative position or size ratio of the two structures can be determined. This relative position or size ratio is then compared to a preset range to achieve quantification. If it falls within the preset range, it is represented by 1; otherwise, it is represented by 0. It should be noted that 0 and 1 here are only examples; other representation methods can be used adaptively in actual implementation. 2) Feature vectorization is performed on the feature relationships and geometric relationships respectively to obtain the corresponding feature vectors.

[0083] When integrating feature relationships and geometric relationships, arithmetic operations such as addition or multiplication can be used to fuse them, thereby obtaining the correlation information between two different structures.

[0084] S103. Integrate the associated information into the feature information to correct the feature information through feature fusion, and obtain the corrected feature information.

[0085] In this embodiment, the purpose of fusing correlation information into feature information is to correct the structural feature information based on the relationships between different structures in the correlation information. This correction can involve deleting, weakening, or modifying the feature information. For example, based on the correlation information, features corresponding to a small structure that should not exist can be deleted from the feature information, or the structural content features corresponding to a large structure can be adjusted. In some cases, large structures that should not exist can also be deleted. For example, if the upper abdominal structure and the hand structure in an ultrasound scan cross-sectional image are considered as two different large structures, the correlation between the upper abdominal structure and its internal small structures and the hand structure is relatively low. Therefore, the correlation information related to the hand is very weak. In this way, the hand structure that should not appear can be removed after feature fusion.

[0086] In one specific embodiment of this application, the associated information is fused into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information, including:

[0087] By utilizing feature relationships, the occurrence rationality of feature information is detected, and the rationality detection results are obtained.

[0088] Geometric relationships are used to perform geometric reliability testing on feature information to obtain reliability testing results;

[0089] By combining the results of rationality testing and reliability testing, the feature information is corrected to obtain corrected feature information.

[0090] For ease of description, the above steps will be combined below.

[0091] Since feature relationships can indicate whether there is a connection between structures, the reasonableness of the occurrence of feature information can be checked based on feature relationships, thus obtaining a reasonableness test result. For example, the simultaneous occurrence of the gastric bubble and the spine is reasonable, so the reasonableness test result is reasonable; while the simultaneous occurrence of the femoral head and the gastric bubble is unreasonable, so the reasonableness test result is unreasonable.

[0092] Since geometric relationships represent the relative positions and sizes of structures, geometric reliability testing can be performed on feature information based on these relationships to obtain reliability test results. For example, the gastric bubble needs to be enclosed by the overall structure of the upper abdomen. Therefore, based on the relationship between the gastric bubble and the overall structure, it can be determined whether the gastric bubble and the overall structure of the upper abdomen are reliable. If it is determined that the gastric bubble is not within the overall structure of the upper abdomen or that the size of the gastric bubble is greater than or equal to the size of the overall structure of the upper abdomen, then the reliability test result is unreliable.

[0093] After obtaining the rationality test results and the reliability test results, the structural feature information can be corrected based on the rationality test results and the reliability test results to obtain the corrected feature information.

[0094] Specifically, by combining the results of rationality testing and reliability testing, the feature information is corrected to obtain corrected feature information, including:

[0095] If the results of the rationality detection determine that there are target features that should not appear, then the features related to the target features are deleted from the feature information.

[0096] If the reliability test results determine that there are errors in the relative position and / or size relationship in the feature information, then the relative position and / or size relationship in the feature information shall be adjusted.

[0097] The feature information that has undergone feature deletion and content adjustment is identified as the corrected feature information.

[0098] In other words, the results of the rationality test can clearly identify target features that should not appear in the current ultrasound scan section image. These target features can specifically be features of a structure that has been determined to be undesirable through the rationality test. Features of such undesirable structures can be directly removed from the feature information.

[0099] Based on the reliability test results, features with errors in at least one of the relative position or size relationship in the feature information can be adjusted accordingly. For example, if the feature information of two structures indicates that their relative positions are incorrect, then the relevant geometric features in these two structures can be adjusted.

[0100] In this way, corrective feature information can be obtained. Low corrective feature information of the current structure indicates that the correlation between the current structure and other structures is weak.

[0101] S104. Using the correction feature information, output the structural identification information corresponding to the overall structure and the internal structure respectively.

[0102] After obtaining the corrected feature information, the structural identification information corresponding to the overall features and the internal structure can be output based on the corrected feature information.

[0103] The structural identification information can include at least location information and content information. Furthermore, location information can be labeled using rectangles, masks, arrows, outlines, etc., while content information can be labeled using structure names, structure numbers, dimensional measurement results, etc. The location and content information can be determined by a deep learning model. In some cases, different thicknesses and colors can be used to output the structural identification information to highlight important structures and downplay less important ones.

[0104] The system can analyze the corrective feature information of each overall structure and internal structure, determine the corresponding structural identification information based on the analysis results, and then output and display it. For example, the corrective feature information is compared with a preset feature information threshold, and structures below the threshold are deleted. In this way, structures with little correlation to other structures can be removed from the ultrasound scan cross-sectional image, improving the detection accuracy. Furthermore, the overall structure and internal structure can be displayed differently based on the range of the corrective feature information. For example, a first feature information threshold and a second feature information threshold are preset, where the first threshold is greater than the second threshold. Structures with corrective feature information greater than the first threshold are highlighted, structures with corrective feature information greater than the second threshold but less than the first threshold are displayed normally, and structures with corrective feature information less than the second threshold are displayed with weakened or no information.

[0105] Using the method provided in the embodiments of this application, feature extraction is performed on the ultrasonic scanning section image to obtain feature information corresponding to the overall structure and the internal structure respectively; the feature information includes structural position features and structural content features; the correlation relationship analysis is performed on the structural position features and structural content features between different structures to obtain correlation information; the correlation information is fused into the feature information to correct the feature information through feature fusion to obtain corrected feature information; using the corrected feature information, the structural identification information corresponding to the overall structure and the internal structure respectively is output.

[0106] In this application, feature extraction is first performed on the ultrasonic scanning section image to obtain the structural location features and structural content features corresponding to the overall structure and internal structure, respectively. Correlation analysis is then performed on the structural location features and structural content features to obtain correlation information. This correlation information is then fused into the feature information, thereby correcting the feature information through feature fusion to obtain corrected feature information. In other words, the correlation information obtained through correlation analysis can fuse feature information from different structures, correcting the structural feature information and resulting in more accurate output of the overall and internal structural location and content information based on the corrected feature information.

[0107] The technical effect of this application is that by analyzing the correlation between the position and content of different structures, the correlation information is integrated into the feature information, and the feature information of the structure is corrected based on the feature correlation between different structures, thereby improving the accuracy of detecting each structure.

[0108] It should be noted that, based on the above embodiments, the embodiments of this application also provide corresponding improvement schemes. In the preferred / improved embodiments, the same or corresponding steps as in the above embodiments can be referred to each other, and the corresponding beneficial effects can also be referred to each other; however, these will not be elaborated upon in the preferred / improved embodiments herein.

[0109] In one specific embodiment of this application, feature relationships and geometric relationships are integrated to obtain correlation information between pairs of different structures, including:

[0110] The characteristic relationships and geometric relationships are quantified separately to obtain the quantification results;

[0111] Based on the quantization results, the correlation weights between each structure and other structures are calculated to obtain the correlation information between different structures.

[0112] Accordingly, the associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including:

[0113] The corrected feature information for each structure is obtained by weighted summation of all associated weights for each structure.

[0114] For ease of description, the above steps will be combined below.

[0115] That is, the feature relationships and geometric relationships are quantified, and the weights between a certain structure A and other structures are obtained based on the quantization results (similar to the connection between a point and other points, with each connection corresponding to a weight). Then, the feature information of structure A is weighted and summed based on these weights to obtain the final feature corresponding to structure A, so as to enhance / weaken / delete the feature.

[0116] To further illustrate: If structures A, B, C, and D exist, by performing correlation analysis on the feature information of each pair of different structures, we can obtain the feature relationships and geometric relationships between each pair of structures. Taking structure A as an example, we can obtain the feature relationships and geometric relationships between A and B, A and C, and A and D, respectively. Based on these feature relationships and geometric relationships, we can quantify the correlation weights (specifically, correlation values) between A and other structures. When performing feature correction processing on structure A, we perform a weighted summation of the feature information of structure A based on these correlation weights, thereby obtaining the corrected feature information of structure A.

[0117] For example, the weighted summation of the feature information of the current structure can be performed as follows: 1) Determine the total number of structures s in the ultrasound scan section image, subtract 1 from the total number of structures s (subtracting the current structure itself), then divide the feature information of the current structure by (s-1) to uniformly divide it into s-1 parts of refined feature information, then multiply each refined feature information by its respective weight, and then sum the results to obtain the weighted summation of the current structure, which is the corrected feature information of the current structure. 2) Normalize each weight, sum the normalized weights to obtain the total weight, and then multiply the total weight by the feature information of the current structure to obtain the corrected feature information of the current structure. Of course, the weighted summation can also be implemented in other ways, and this application embodiment does not limit this.

[0118] In some embodiments, the rationality test results and reliability test results can be quantified separately to determine the weights between the pairs of structures, and then a weighted summation process can be performed.

[0119] For example, after obtaining structural location features and structural content features, the structural location features and structural content features are interacted with. Specifically, this may involve analyzing their features (such as location, size, shape, texture, etc.) and determining how they are related to each other, using correlation parameter values ​​to represent them (specifically, they may be obtained based on the fusion of geometric relationships and feature relationships), so that the obtained correlation information can be fused back into the feature information in the future.

[0120] One specific implementation of determining correlation can be to enhance / weaken the representation of each structure by calculating the relationship features between pairs of structures (including appearance features and geometric (positional) features). Typical implementations include models based on graph convolutional networks or attention mechanisms. For example, the umbilical vein, stomach bubble, and spine will appear simultaneously in a standard fetal abdominal cross-section, and their relative positions are relatively fixed. Therefore, this relationship can be strengthened to improve the expressive power of the features.

[0121] In one specific embodiment of this application, correlation analysis is performed on the structural positional features and structural content features between different structures to obtain correlation information, including:

[0122] Construct a relationship matrix using structural location features and structural content features;

[0123] Using the relation matrix, calculate the correlation parameter values ​​between every two structures;

[0124] The correlation parameter values ​​between any two structures are defined as the correlation information.

[0125] Specifically, the correlation analysis between structural location features and structural content features can be performed using a structural interaction model. This model corresponds to deep learning techniques such as attention mechanisms and graph convolutional networks that can construct relationship matrices. This relationship matrix calculates the relationships between each structure and other structures based on structural location information (corresponding to structural location features) and content information (corresponding to structural content features). For example, the model calculates and weights the feature and geometric relationships between the gastric bubble and the umbilical vein, spine, overall upper abdominal structures, and femoral head, enhancing the features of the gastric bubble. Similarly, misidentified femoral head structures, after the relationship matrix calculation, will show very low correlations with other structures and will therefore be removed.

[0126] In one specific embodiment of this application, the associated information is fused into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information, including:

[0127] The association information and feature information are input into the fully connected layer for fully connected processing to obtain fully connected feature information;

[0128] The fully connected feature information is identified as the correction feature information.

[0129] Fully connected layers allow for full interaction between related and feature information, enabling the output fully connected feature information to fully integrate related information. This allows for the removal or adjustment of some feature information, resulting in highly accurate corrected feature information.

[0130] In one specific embodiment of this application, the associated information is fused into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information, including:

[0131] Convolutional calculations are performed on the correlation information and feature information to obtain convolutional feature information;

[0132] The convolutional feature information is identified as the correction feature information.

[0133] Convolutional computation can fully integrate correlation and feature information, allowing the output convolutional feature information to fully incorporate correlation information, thereby removing or adjusting some feature information, and finally obtaining corrected feature information with high accuracy.

[0134] In some embodiments, the fully connected processing and convolution processing described above can also be implemented together in a single model.

[0135] In practical applications, the associated information is fused back into the feature information of the object through fully connected and / or convolutional operations, and finally the specific category and location information of each structure is output.

[0136] The following section will take the application of the above-mentioned ultrasound image recognition method in obstetric ultrasound examination as an example to provide a detailed explanation of the practical application of this ultrasound image recognition method.

[0137] The purpose of this invention is to use deep learning methods to model the relationships between various obstetric cross-sections, structural appearances, and shapes, thereby improving the accuracy of structural detection and enhancing quality control.

[0138] Specifically, based on a deep learning model, the positional and appearance information of each structure can be interacted. Please refer to [reference needed]. Figure 2 The workflow is as follows:

[0139] 1. The ultrasound equipment begins scanning and acquires image information from the doctor's imaging, which includes obstetric sections.

[0140] Second, image information is analyzed and processed using deep learning models.

[0141] For ease of description, the internal process of this deep learning model will be broken down into two stages of detection model for explanation.

[0142] (1) The first-stage model detects the overall structure and internal small structure corresponding to the image information and determines the feature information of the overall structure and small structure. This result is a prediction and has a low accuracy.

[0143] (2) The two-stage model performs interactive analysis on different structures based on the location and content information detected in the first stage. The interactive analysis process specifically involves analyzing the features of each structure (such as location, size, shape, texture, etc.) and determining how they are related to each other. Then, the obtained association information is fused back into the feature representation of the structure.

[0144] Specifically, determining the correlation can be achieved by using a model based on graph convolutional networks or attention mechanisms to calculate the relationship features (including appearance features and geometric (positional) features) between pairs of structures to enhance / weaken the representation of each structure. For example, the umbilical vein, stomach bubble, and spine will appear simultaneously in a standard fetal abdominal cross-section, and their relative positions are relatively fixed. Therefore, this module will strengthen this relationship and improve the expressive power of the features.

[0145] The following explanation of the model's internal logic and principles, based on its general structure, will be provided:

[0146] Using images as input, the first stage of the model is to extract feature information. The feature extraction model is a deep learning model such as ResNet, VGG, and ViT, which extracts the appearance features and geometric (positional) features corresponding to large structures and small structures, respectively.

[0147] After the first stage outputs coarse structural location information (feature information), the corresponding content information is obtained based on this information, and the location and content information of all structures are interacted through a structural interaction model. This structural interaction model is based on deep learning techniques such as attention mechanisms and graph convolutional networks, which can construct relationship matrices.

[0148] Furthermore, the relation matrix in the structural interaction model calculates the relationship between each structure output in the first stage and other structures to obtain relation weights for subsequent feature fusion.

[0149] For example, after feature extraction from the corresponding ultrasound scan image of the abdomen, the extracted structures include the stomach bubble, umbilical vein, spine, overall upper abdominal structure, and femoral head. For the stomach bubble, a relational matrix is ​​used to calculate its feature and geometric relationships with the umbilical vein, spine, overall upper abdominal structure, and femoral head, and then weighted to enhance the structure's features. Furthermore, misidentified bone structures, after relational matrix calculation, will show very low correlation with other structures and are therefore ultimately discarded.

[0150] The relation weights obtained in the previous step are fused with the appearance features of each structure to obtain relation information. Fusion methods include, but are not limited to, weighted summation after linear transformation. After enhancement, the features of each structure not only contain its own information but also its relation information with other structures.

[0151] (3) Correct the feature information of the overall structure and each substructure.

[0152] The specific implementation of this step can be: fusing the relational information back into the feature representation of the structure through operations such as fully connected or convolution.

[0153] 3. If the scanning is still ongoing, return to the step of obtaining the obstetric section; if the scanning is finished, confirm that the identification is complete.

[0154] IV. Output the specific category and location information of each structure in the corresponding section.

[0155] Please refer to Figure 3 The image shows an example of structural detection in the fetal brain. Comparing the proposed output (including: thalamic balance section, lateral brain horizontal section, cavum pellucidum, thalamus, and lateral ventricle) before structural interaction (i.e., the process of determining the relationships between structures and fusing feature information), with the output after structural interaction (including: lateral brain horizontal section, cavum pellucidum, thalamus, and lateral ventricle), it can be observed that the actual output removes the thalamic horizontal section (i.e., the dashed box in the image), which should not have appeared. In other words, by performing relationship calculations and information interaction on various key structures in the obstetric section, the accuracy of detecting the obstetric section and its internal key structures is improved.

[0156] It should be noted that, in order to make it easier to see the differences in the recognition results, in Figure 3 The corresponding output does not overlay the recognition result onto the input cross-sectional image. In practical applications, when displaying the output results, the recognition result can be overlaid onto the input cross-sectional image.

[0157] Corresponding to the above method embodiments, this application also provides an ultrasound image recognition device. The ultrasound image recognition device described below can be referred to in correspondence with the ultrasound image recognition method described above.

[0158] See Figure 4 As shown, the device includes the following modules:

[0159] Feature extraction module 101 is used to extract features from ultrasound scan cross-sectional images to obtain feature information corresponding to the overall structure and internal structure, respectively; the feature information includes structural location features and structural content features;

[0160] The interactive association module 102 is used to analyze the association relationship between the structural position features and structural content features of different structures to obtain association information.

[0161] Feature correction module 103 is used to fuse the associated information into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information;

[0162] The structure detection module 104 is used to output the structure identification information corresponding to the overall structure and the internal structure respectively using the correction feature information.

[0163] Using the apparatus provided in this application embodiment, feature extraction is performed on the ultrasonic scanning section image to obtain feature information corresponding to the overall structure and the internal structure, respectively. The feature information includes structural position features and structural content features. The correlation relationship between the structural position features and the structural content features of different structures is analyzed to obtain correlation information. The correlation information is fused into the feature information to correct the feature information through feature fusion to obtain corrected feature information. Using the corrected feature information, structural identification information corresponding to the overall structure and the internal structure is output.

[0164] In this application, feature extraction is first performed on the ultrasonic scanning section image to obtain structural location features and structural content features corresponding to the overall structure and internal structure, respectively. Correlation analysis is then performed on the structural location features and structural content features to obtain correlation information. This correlation information is then fused into the feature information, thereby correcting the feature information through feature fusion to obtain corrected feature information. In other words, the correlation information obtained through correlation analysis can fuse feature information from different structures, correcting the structural feature information and resulting in more accurate output of structural identification information corresponding to the overall structure and internal structure based on the corrected feature information.

[0165] The technical effect of this application is that by analyzing the correlation between the position and content of different structures, the correlation information is integrated into the feature information, and the feature information of the structure is corrected based on the feature correlation between different structures, thereby improving the accuracy of detecting each structure.

[0166] In one specific embodiment of this application, the interactive association module is specifically used to perform association analysis on pairs of different structures based on the structural position features and the structural content features to obtain the association information of pairs of different structures; wherein, different structures correspond to the overall structure and the internal structure, or, different structures correspond to different organizational structures in the internal structure.

[0167] In one specific embodiment of this application, the interactive association module includes: a position feature analysis submodule, used to perform association analysis on the position features of two different structures to obtain the feature relationship between the two different structures; a content feature analysis submodule, used to perform association analysis on the content features of two different structures to obtain the geometric relationship between the two different structures; and a feature relationship integration submodule, used to integrate the feature relationship and the geometric relationship to obtain the association information between the two different structures.

[0168] In one specific embodiment of this application, the feature relationship indicates whether there is an association between structures; the geometric relationship indicates the relative position and size relationship between structures. Correspondingly, the interactive association module includes: a rationality detection submodule, used to perform rationality detection on the feature information using the feature relationship to obtain a rationality detection result; a reliability detection submodule, used to perform geometric reliability detection on the feature information using the geometric relationship to obtain a reliability detection result; and a feature correction submodule, used to combine the rationality detection result and the reliability detection result to correct the feature information to obtain the corrected feature information.

[0169] In one specific embodiment of this application, the feature correction submodule includes: a feature deletion unit, configured to delete features related to the target feature from the feature information if the reasonableness detection result determines that there is a target feature that should not appear; a content adjustment unit, configured to adjust the relative position and / or size relationship in the feature information if the reliability detection result determines that there is an error in the relative position and / or size relationship in the feature information; and an information determination unit, configured to determine the feature information after feature deletion and content adjustment as the corrected feature information.

[0170] In one specific embodiment of this application, the feature relationship integration submodule includes: a quantization unit, used to quantize the feature relationship and the geometric relationship respectively to obtain a quantization result; a weight determination unit, used to calculate the correlation weight between each structure and other structures based on the quantization result to obtain the correlation information between pairs of different structures; correspondingly, a feature correction module, used to perform a weighted summation of the corresponding feature information based on all the correlation weights of each structure to obtain the corrected feature information corresponding to each structure.

[0171] In one specific embodiment of this application, the interactive association module includes: a matrix construction submodule, used to construct a relationship matrix using the structural position features and the structural content features; an association parameter calculation submodule, used to calculate the association parameter value between every two structures using the relationship matrix; and an association information determination submodule, used to determine the association parameter value between every two structures as the association information.

[0172] In one specific embodiment of this application, the ultrasound image recognition method is implemented using a deep learning model, which includes fully connected layers and / or convolutional layers; a feature correction module is specifically used to input the association information and the feature information into the fully connected layer for fully connected processing, and determine the output of the fully connected layer as the corrected feature information; or, the feature correction module is specifically used to input the association information and the feature information into the convolutional layer for convolution calculation, and determine the output of the convolutional layer as the corrected feature information.

[0173] Corresponding to the above method embodiments, this application also provides an electronic device. The electronic device described below can be referred to in conjunction with the ultrasound image recognition method described above.

[0174] See Figure 5 As shown, the electronic device includes:

[0175] Memory 332 is used to store computer programs;

[0176] The processor 322 is used to implement the steps of the ultrasound image recognition method of the above method embodiment when executing a computer program.

[0177] For details, please refer to Figure 6 , Figure 6 This is a schematic diagram of the specific structure of an electronic device provided in this embodiment. The electronic device can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be temporary or permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the electronic device 301.

[0178] Electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0179] The steps in the ultrasound image recognition method described above can be implemented by the structure of an electronic device.

[0180] Corresponding to the above method embodiments, this application also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the ultrasound image recognition method described above.

[0181] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the ultrasound image recognition method described in the above method embodiments.

[0182] The readable storage medium can specifically be a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or any other readable storage medium capable of storing program code.

[0183] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0184] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0185] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0186] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0187] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An ultrasound image recognition method, characterized in that, include: Feature extraction is performed on the ultrasonic scanning section images to obtain feature information corresponding to the overall structure and internal structure, respectively. The feature information includes structural location features and structural content features; The correlation information is obtained by analyzing the relationship between the structural position features and structural content features of different structures. The associated information is fused into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information; Using the corrective feature information, output the structural identification information corresponding to the overall structure and the intrinsic structure respectively.

2. The method according to claim 1, characterized in that, A correlation analysis is performed on the structural positional features and structural content features of different structures to obtain correlation information, including: Based on the structural location features and structural content features, correlation analysis is performed on pairs of different structures to obtain correlation information between pairs of different structures; wherein, different structures correspond to the overall structure and the internal structure, or, different structures correspond to different organizational structures within the internal structure.

3. The method according to claim 2, characterized in that, Based on the structural location features and structural content features, correlation analysis is performed on pairs of different structures to obtain correlation information between pairs of different structures, including: Correlation analysis is performed on the structural positional features of two different structures to obtain the feature relationships between the two different structures; By performing correlation analysis on the structural content features of two different structures, the geometric relationships between the two different structures are obtained; By integrating the feature relationships and the geometric relationships, we can obtain the association information of two different structures.

4. The method according to claim 3, characterized in that, The characteristic relationships indicate whether there is a correlation between structures; the geometric relationships indicate the relative positions and sizes of the structures. Accordingly, the associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including: Using the aforementioned feature relationships, the occurrence rationality of the feature information is detected, and a rationality detection result is obtained. Using the geometric relationship, geometric reliability detection is performed on the feature information to obtain reliability detection results; By combining the rationality test results and the reliability test results, the feature information is corrected to obtain the corrected feature information.

5. The method according to claim 4, characterized in that, Combining the rationality test results and the reliability test results, the feature information is corrected to obtain the corrected feature information, including: If the rationality detection results determine that there is a target feature that should not appear, then the feature related to the target feature is deleted from the feature information; If the reliability test results determine that there are errors in the relative position and / or size relationship in the feature information, then the relative position and / or size relationship in the feature information shall be adjusted. The feature information after feature deletion and content adjustment is determined as the corrected feature information.

6. The method according to claim 3, characterized in that, Integrating the feature relationships and the geometric relationships yields pairwise association information of different structures, including: The feature relationship and the geometric relationship are quantized respectively to obtain the quantization results; Based on the quantification results, the correlation weights between each structure and other structures are calculated to obtain the correlation information between different structures. Accordingly, the associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including: The corrected feature information for each structure is obtained by weighted summation of all associated weights for each structure.

7. The method according to any one of claims 1 to 6, characterized in that, A correlation analysis is performed on the structural positional features and structural content features of different structures to obtain correlation information, including: A relationship matrix is ​​constructed using the structural location features and the structural content features; Using the aforementioned relationship matrix, calculate the correlation parameter values ​​between every two structures; The correlation parameter values ​​between every two structures are determined as the correlation information.

8. The method according to any one of claims 1 to 6, characterized in that, The ultrasound image recognition method is implemented using a deep learning model, which includes fully connected layers and / or convolutional layers. The associated information is fused into the feature information to correct the feature information through feature fusion, resulting in corrected feature information, including: The association information and the feature information are input into a fully connected layer for fully connected processing, and the output of the fully connected layer is determined as the corrected feature information. or, The associated information and the feature information are input into a convolutional layer for convolution calculation, and the output of the convolutional layer is determined as the corrected feature information.

9. An ultrasonic image recognition device, characterized in that, include: The feature extraction module is used to extract features from the ultrasound scan section image to obtain feature information corresponding to the overall structure and the internal structure, respectively. The feature information includes structural location features and structural content features; The interactive association module is used to analyze the association relationship between the structural position features and structural content features of different structures to obtain association information; A feature correction module is used to fuse the associated information into the feature information to correct the feature information through feature fusion, thereby obtaining corrected feature information; The structure detection module is used to output the structural identification information corresponding to the overall structure and the internal structure respectively using the correction feature information.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the ultrasound image recognition method as described in any one of claims 1 to 8 when executing the computer program.

11. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the ultrasound image recognition method as described in any one of claims 1 to 8.

Images