A detection method and related device for thyroid cancer or breast cancer

Abstract

The present application discloses a method and related device for detecting thyroid cancer or breast cancer, the method comprising acquiring a region of interest of a target image to obtain an image of interest; determining a number of weighted feature maps and contour feature maps of the target image based on the image of interest; determining the benign and malignant nodule category and nodule feature category corresponding to the target image based on the number of weighted feature maps; and determining the nodule position and size information of the target image based on the contour feature map. The present application implements fully automatic output from an image to classification and segmentation results, and the segmentation results include attribute information such as benign and malignant nodule category, nodule feature category, and location and size, which can improve the detection speed of thyroid cancer or breast cancer. At the same time, information such as nodule feature category and location and size is conducive to the classification of nodule benign and malignant categories, thereby improving the reliability of the detection results of thyroid cancer or breast cancer.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method and related device for detecting thyroid cancer or breast cancer. Background Art

[0002] Thyroid cancer is the seventh most common cancer in women and the fifteenth most common cancer in men. Breast cancer has the highest morbidity and mortality rate among women. These two cancers pose a serious threat to human health and place a heavy burden on individuals and families. Ultrasound, due to its non-invasive, real-time, and inexpensive nature, has become the primary imaging modality for thyroid and breast (thyroid and breast) detection. The medical community has developed the Thyroid Imaging Reporting and Data System (TI-RADS) and the Breast Imaging Reporting and Data System (BI-RADS) as ultrasound imaging diagnostic guidelines to standardize diagnoses. Clinicians should first characterize nodules based on their shape, proportions, borders, margins, echogenicity, calcifications, and other characteristics. However, the implementation of these guidelines is inevitably subject to subjectivity, and their accuracy depends on the experience and skills of the clinician.

[0003] Therefore, the industry has proposed computer-aided diagnosis (CAD) systems to help doctors diagnose thyroid and breast diseases. In recent years, with the popularization and advancement of deep learning, thyroid and breast nodule diagnosis methods represented by deep learning have gradually gained attention. However, current thyroid and breast CAD systems generally have limited functionality and are inaccurate in determining whether nodules are benign or malignant, resulting in low doctor confidence in CAD system examination results.

[0004] Therefore existing technology still needs to be improved and improved. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a method and related device for detecting thyroid cancer or breast cancer in response to the shortcomings of the existing technology.

[0006] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides a method for detecting thyroid cancer or breast cancer, the method comprising:

[0007] Acquiring a region of interest of a target image to obtain an image of interest, wherein the target image includes a thyroid image or a breast image, and the target image includes at least one nodule region;

[0008] Determining a plurality of weighted feature maps and contour feature maps of the target image according to the image of interest;

[0009] Determining the benign or malignant category of the nodule and the nodule feature category corresponding to the target image according to the plurality of weighted feature maps;

[0010] The nodule position and size attributes of the target image are determined according to the contour feature map.

[0011] The method for detecting thyroid cancer or breast cancer, wherein the several weighted feature maps and contour feature maps are determined by a preset detection network model, wherein the preset detection network model includes a feature extraction module, an attention module and a fusion module, the input item of the attention module includes the output item of the feature extraction module, and the input item of the fusion module includes the output item of the feature extraction module and the output item of the attention module.

[0012] The method for detecting thyroid cancer or breast cancer, wherein determining the weighted feature map and the contour feature map of the target image based on the image of interest specifically includes:

[0013] Inputting the image of interest into a feature extraction module, and outputting a feature map through the feature extraction module;

[0014] Input the feature map into the attention module, and output a number of activation maps and contour feature maps through the attention module;

[0015] The activation maps and the feature maps are input into a fusion module, and the fusion module outputs a plurality of weighted feature maps.

[0016] The method for detecting thyroid cancer or breast cancer, wherein the preset detection network model includes a multi-attribute classification module, and determining the benign and malignant nodule category and the nodule feature category corresponding to the target image based on the plurality of weighted feature maps specifically includes:

[0017] The weighted feature maps are input into a multi-attribute classification module, and the multi-attribute classification module determines the sub-attribute features corresponding to the target image, and determines the benign or malignant nodule category and the nodule feature category corresponding to the target image based on the sub-attribute features.

[0018] The method for detecting thyroid cancer or breast cancer, wherein the loss function adopted in the training process of the preset detection network model includes a classification loss term, an attention loss term and a segmentation loss term, wherein the classification loss term is determined based on the predicted categories and category labels corresponding to the training image determined by the multi-attribute classification module, the attention loss term is determined based on the candidate categories and category labels corresponding to the training image determined by the attention module, and the segmentation loss term is determined based on the contour feature map and the contour mask map.

[0019] The method for detecting thyroid cancer or breast cancer, wherein determining the nodule location and size attributes of the target image based on the contour feature map specifically includes:

[0020] Determining the regional position, the length of the major axis, and the length of the minor axis of the nodule region according to the contour feature map, wherein the major axis and the minor axis are perpendicular to each other;

[0021] The region position, major axis length, and minor axis length are used as nodule position and size attributes of the target image.

[0022] In the method for detecting thyroid cancer or breast cancer, the step of acquiring a region of interest of a target image to obtain an image of interest specifically comprises:

[0023] Converting the target image into a grayscale image, and converting the grayscale image into a binary image;

[0024] The maximum connected region of the binary image is obtained, and a region of interest of the target image is determined based on the maximum connected region and the target image to obtain an image of interest.

[0025] A second aspect of the embodiments of the present application provides a device for detecting thyroid cancer or breast cancer, the device comprising:

[0026] an acquisition module, configured to acquire a region of interest of a target image to obtain an image of interest, wherein the target image includes a thyroid image or a breast image, and the target image includes at least one nodule region;

[0027] A feature determination module, configured to determine a plurality of weighted feature maps and a contour feature map of the target image based on the image of interest;

[0028] a classification module, configured to determine, based on the weighted feature maps, a benign or malignant nodule category and a nodule feature category corresponding to the target image;

[0029] The attribute determination module is used to determine the nodule position and size attributes of the target image according to the contour feature map.

[0030] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the above-described methods for detecting thyroid cancer or breast cancer.

[0031] A fourth aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor;

[0032] The communication bus realizes the connection and communication between the processor and the memory;

[0033] When the processor executes the computer-readable program, the steps in any of the above-described methods for detecting thyroid cancer or breast cancer are implemented.

[0034] Beneficial effects: Compared with the prior art, the present application provides a method and related apparatus for detecting thyroid cancer or breast cancer, the method comprising acquiring a region of interest of a target image to obtain an image of interest; determining a number of weighted feature maps and contour feature maps of the target image based on the image of interest; determining the benign and malignant nodule category and nodule feature category corresponding to the target image based on the number of weighted feature maps; and determining the nodule position and size attributes of the target image based on the contour feature map. The present application realizes fully automatic output from image to classification and segmentation results, and the segmentation results include attribute information such as benign and malignant nodule category, nodule feature category, and position and size, which can improve the detection speed of thyroid cancer or breast cancer. At the same time, information such as nodule feature category and position and size is conducive to the classification of benign and malignant nodule category, thereby improving the reliability of the detection results of thyroid cancer or breast cancer. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0036] Figure 1 This is a flow chart of the method for detecting thyroid cancer or breast cancer provided in this application.

[0037] Figure 2 This is a flow chart of the principles of the method for detecting thyroid cancer or breast cancer provided in this application.

[0038] Figure 3 This is a flow chart of the principles of the process of acquiring images of interest in the method for detecting thyroid cancer or breast cancer provided in this application.

[0039] Figure 4 This is a schematic diagram of the model structure of the preset detection network model in the thyroid cancer or breast cancer detection method provided in this application.

[0040] Figure 5 This is a structural principle diagram of an embodiment of the attention module in the thyroid cancer or breast cancer detection method provided in this application.

[0041] Figure 6 This is a structural principle diagram of another embodiment of the attention module in the thyroid cancer or breast cancer detection method provided in the present application.

[0042] Figure 7 This is a schematic diagram of the structural principle of the thyroid cancer or breast cancer detection device provided in this application.

[0043] Figure 8 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0044] This application provides a method and related device for detecting thyroid cancer or breast cancer. To make the purpose, technical solution, and effects of this application more clear and explicit, the application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate this application and are not intended to limit this application.

[0045] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0046] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0047] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not imply the order of execution. The order of execution of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0048] After research, the inventors found that thyroid cancer is the seventh most common cancer in women and the fifteenth most common cancer in men, and breast cancer is the cancer with the highest morbidity and mortality in women. These two cancers seriously threaten human health and impose a heavy burden on individuals and families. Ultrasound has become the primary imaging method for thyroid and breast (thyroid and breast) detection due to its non-invasive, real-time, and inexpensive characteristics. The medical community has proposed the Thyroid Imaging Reporting and Data System (TI-RADS) and the Breast Imaging Reporting and Data System (BI-RADS) as ultrasound imaging diagnostic guidelines to enable doctors to standardize their diagnoses as much as possible. Clinicians should first characterize nodules based on their shape, proportion, boundaries, edges, echogenicity, calcifications, and other characteristics. However, the implementation of the guidelines is inevitably affected by subjectivity, and their accuracy also depends on the experience and skills of the clinicians.

[0049] Therefore, the industry has proposed computer-aided diagnosis (CAD) systems to help doctors diagnose thyroid and breast diseases. In recent years, with the popularization and advancement of deep learning, thyroid and breast nodule diagnosis methods represented by deep learning have gradually gained attention. However, current thyroid and breast CAD systems generally have limited functionality and are inaccurate in determining whether nodules are benign or malignant, resulting in low doctor confidence in CAD system examination results.

[0050] In order to solve the above problems, in an embodiment of the present application, a region of interest of a target image is acquired to obtain an image of interest; a number of weighted feature maps and contour feature maps of the target image are determined based on the image of interest; the benign and malignant nodule category and the nodule feature category corresponding to the target image are determined based on the several weighted feature maps; and the nodule position and size attributes of the target image are determined based on the contour feature map. The present application realizes fully automatic output from image to classification and segmentation results. The segmentation results include attribute information such as benign and malignant nodule category, nodule feature category, and position and size, which can improve the detection speed of thyroid cancer or breast cancer. At the same time, information such as nodule feature category and position and size is conducive to the classification of benign and malignant nodule category, thereby improving the reliability of the detection results of thyroid cancer or breast cancer.

[0051] The application content will be further explained below through description of embodiments in conjunction with the accompanying drawings.

[0052] This embodiment provides a method for detecting thyroid cancer or breast cancer, such as Figure 1 and Figure 2 As shown, the method includes:

[0053] S10: Acquire a region of interest of a target image to obtain an image of interest.

[0054] Specifically, the target image includes a thyroid image or a breast image, and the target image is an ultrasound image and carries a nodule area, that is, the target image can be a thyroid ultrasound image carrying a thyroid nodule area, or a breast ultrasound image carrying a breast nodule area. It can be understood that the thyroid cancer or breast cancer detection method provided in this embodiment can use the thyroid ultrasound image as the target image to determine the classification result of thyroid cancer, or can use the breast ultrasound image as the target image to determine the classification result of breast cancer. In addition, the target image can include one nodule area or multiple nodule areas. When multiple nodule areas are included, each nodule area can be determined using the method provided in this embodiment. This embodiment is explained by taking one nodule area as an example.

[0055] An ultrasound image may include several functional areas, such as an image area with the anatomical structure of the lesion, an area recording the scan time, an area recording the patient's name, an area recording the hospital identification, and an area recording the current ultrasound parameters. Among these functional areas, only the image area with the anatomical structure of the lesion carries valid information, while the other functional areas carry invalid information, and invalid information will affect the accuracy of the subsequent determination of the benign or malignant category of the nodule, the nodule feature category, and the nodule location and size attributes. Therefore, after acquiring the target image, the image area with the anatomical structure of the lesion in the ultrasound image can be first acquired, and the image area with the anatomical structure of the lesion can be used as the region of interest, so that when subsequently determining the benign or malignant category of the nodule, the nodule feature category, and the nodule location and size attributes, the image information acquired is valid information.

[0056] Based on this, after acquiring the target image, the region of interest of the target image will be acquired, and the image content of the image area other than the region of interest in the target image will be cleared to obtain the image of interest. Among them, the currently commonly used method for acquiring the region of interest is to determine it by cropping a fixed area. For example, for any 1260*820 thyroid ultrasound image, a fixed cropping of a specified area is performed to determine the image of interest. Although this method can acquire image areas with lesion anatomical structures in some ultrasound images, in the face of complex ultrasound usage scenarios, fixed cropping of a specified area may cause the loss of effective information, thereby affecting the accuracy of subsequent determination of the benign and malignant category of nodules, the category of nodules features, and the detection of nodule position and size attributes.

[0057] Based on this, in one implementation of this embodiment, as Figure 3 As shown, acquiring the region of interest of the target image to obtain the image of interest specifically includes:

[0058] Converting the target image into a grayscale image, and converting the grayscale image into a binary image;

[0059] The maximum connected region of the binary image is obtained, and a region of interest of the target image is determined based on the maximum connected region and the target image to obtain an image of interest.

[0060] Specifically, the target image is a three-channel RGB image, the grayscale image is a single-channel image, and the grayscale value of each pixel in the grayscale image ranges from 0 to 255, where 0 is pure black and 255 is pure white. A binary image is an image that includes only two grayscale values. For example, the grayscale value of some pixels in the binary image is 0, and the grayscale value of some pixels is 255. In one specific implementation, the binary image is determined based on a threshold method. That is, the grayscale image is binarized based on the threshold method to obtain the binary image. The process of binarizing the grayscale image based on the threshold method can be: comparing the grayscale value of each pixel in the grayscale image with a preset threshold to obtain all first pixels whose grayscale value is greater than or equal to the preset threshold, and all second pixels whose grayscale value is less than the preset threshold, setting the pixel value of all first pixels to 255, and setting the pixel value of the second pixels to 0, to obtain the binary image, wherein the binary image only includes white areas and black areas.

[0061] The preset threshold is pre-set, and the same preset threshold can be configured for the target images used by ultrasound devices of all ultrasound device models, or different preset thresholds can be configured for ultrasound devices of different device models, so that when determining a binary image, the device model of the ultrasound device corresponding to the target image can be obtained first and the preset threshold is determined based on the device model, and then binarization is performed based on the threshold. However, the inventors have found through research that the number of ultrasound devices that can be applied to a small threshold is greater than the number of ultrasound devices that can be applied to a large threshold. When an appropriately small threshold is selected, the threshold can be applied to ultrasound devices of all ultrasound device models without affecting the binarization result. Therefore, in a typical implementation of this embodiment, the preset threshold is 1, and the preset threshold is used for the target images used by ultrasound devices of all ultrasound device models.

[0062] After acquiring the binary image, the maximum connected region method is used to select the largest connected region in the binary image, and other image regions in the target image except the image region corresponding to the largest connected region are discarded to obtain an image of interest, wherein the largest connected region is the largest white connected region, and the image region in the target image corresponding to the largest connected region is the region of interest in the target image. In addition, after acquiring the image region of interest, the image region of interest can be processed to adjust the image size of the image region of interest to a specified length and width. The processing of the image region of interest can include zero padding around the image, interpolation scaling, cropping, etc., or a combination of these methods.

[0063] S20: Determine several weighted feature maps and contour feature maps of the target image according to the image of interest.

[0064] Specifically, several weighted feature maps are used to determine several nodule benign and malignant categories and nodule feature categories, and contour feature maps are used to determine nodule location and size attributes, wherein the several weighted feature maps and contour feature maps are determined by a preset detection network model. Figure 4 As shown, the preset detection network model includes a feature extraction module, an attention module and a fusion module. The input item of the feature extraction module is the image of interest, the input item of the attention module includes the output item of the feature extraction module, and the input item of the fusion module includes the output item of the feature extraction module and the output item of the attention module.

[0065] Based on this, the step of determining the weighted feature map and the contour feature map of the target image according to the image of interest specifically includes:

[0066] Inputting the image of interest into a feature extraction module, and outputting a feature map through the feature extraction module;

[0067] Input the feature map into the attention module, and output a number of activation maps and contour feature maps through the attention module;

[0068] The activation maps and the feature maps are input into a fusion module, and the fusion module outputs a plurality of weighted feature maps.

[0069] Specifically, the feature extraction module is used to learn the image features of the image of interest. The feature extraction module can obtain a feature map corresponding to the image of interest. The feature map extraction module can use a deep convolutional neural network to output the deep image features of the image of interest. In addition, the deep convolutional neural network can use an existing convolutional neural network structure, such as the VGG network structure, the ResNet network structure, the DenseNet network structure, the MobileNet network structure, and the Inception network structure.

[0070] The attention module is configured with an attention mechanism, through which the feature points with high importance in the feature image can be learned, while the feature points with low importance can be ignored. This is because in the feature map, the importance of each feature point in the feature map is different. For example, in the feature map used for classification tasks, the importance of the feature points for foreground information (such as cats and dogs) in the feature map is higher than the importance of the feature points corresponding to background information (such as grass). Then, the attention mechanism can focus on foreground feature extraction. Thus, the attention module in this embodiment determines the key features required for the benign and malignant categories of nodules, the feature categories of nodules, and the location and size attributes of nodules, so as to obtain several activation maps and contour feature maps.

[0071] Each activation map in the plurality of activation maps is different, and each activation map in the plurality of activation maps corresponds to a classification category included in the benign and malignant nodule category and the nodule feature category that needs to be determined later. It can be understood that the number of the plurality of activation maps is equal to the number of classification categories included in the benign and malignant nodule category and the nodule feature category, and the plurality of activation maps have a one-to-one correspondence with the classification categories included in the benign and malignant nodule category and the nodule feature category, wherein each activation map is used to reflect the attention of its corresponding classification category to each feature point in the feature map.

[0072] In one implementation of this embodiment, the attention module may include a first convolutional network unit, a downsampling unit, a second convolutional network unit, and a global pooling and fully connected unit. The input item of the first convolutional network unit is a feature map. The downsampling unit is parallel to the second convolutional unit. The input items of the downsampling unit and the second convolutional unit are both output items of the first convolutional network unit. The output items of the second convolutional network unit are contour feature maps and several activation maps. The global pooling and fully connected unit are connected to the downsampling unit. The output item of the downsampling unit is the input item of the global pooling and fully connected unit. The output items of the global pooling and fully connected unit are candidate categories of benign and malignant nodules and nodule feature categories. Among them, if Figure 5 As shown, the second convolutional network unit may include only one convolutional network unit 2, through which several activation images and contour feature maps can be output synchronously, or, as shown in FIG. Figure 6 As shown, the second convolutional network unit can be convolutional network unit 2 and convolutional network unit 3, convolutional network unit 2 is used to output a contour feature map, and convolutional network unit 3 is used to input several activation maps.

[0073] The working process of the attention module can be as follows: the attention module inputs the feature map input therein into the first convolutional network unit, extracts the feature map through the first convolutional network unit to obtain the first feature map, and the first feature map is respectively input into the first branch composed of the downsampling unit, the global pooling unit and the fully connected unit, and the second branch composed of the second convolutional network unit, and the candidate categories of the benign and malignant nodules and the nodule feature categories are generated by the first branch, and several contour feature maps based on attention are generated by the second branch. The first branch in the attention module is used to perform the multi-attribute classification task, and the candidate categories determined by the multi-attribute classification task are used to drive the attention module to generate a more accurate activation map. That is to say, when training the preset detection network model provided in the embodiment, the generation of the activation map can be supervised by the candidate categories obtained by the first grouping for performing the multi-attribute classification task, thereby improving the accuracy of the activation map feature positioning, thereby improving the module performance of the attention module, and thus providing the network performance of the preset convolutional network. Of course, in actual applications, the first branch can be directly used to generate several activation maps and contour feature maps. In addition, it is worth noting that the attention module can only include the first branch and not the second branch.

[0074] The fusion module is used to fuse the feature map with each activation map. The fusion module includes several fusion units, and the fusion units are combined with several activation maps. Figure 1 One by one, each fusion unit is used to fuse its corresponding activation map and feature map to obtain the weighted feature map corresponding to the activation map. Of course, in practical applications, the fusion module can also include a fusion unit, each activation map is sequentially input into the fusion unit, and fused with the feature map input into the fusion unit to obtain the weighted feature map corresponding to each activation map, thereby obtaining several weighted feature maps. It can be seen that several weighted feature maps are combined with several activation maps. Figure 1 One by one, each weighted feature map is obtained by fusion of its corresponding activation map and feature map, where the fusion of activation map and feature map refers to the multiplication of activation map and feature map.

[0075] S30. Determine the benign or malignant nodule category and the nodule feature category corresponding to the target image according to the plurality of weighted feature maps.

[0076] Specifically, the nodule benign and malignant category includes a classification category for reflecting the benign and malignant nature of the nodule, and the nodule feature category includes a classification category for reflecting the attribute characteristics of the nodule, wherein the nodule benign and malignant category can include one or more classification categories, and the nodule feature category can include one or more classification categories.

[0077] In a typical implementation of this embodiment, as shown in Table 1, when the target image is a thyroid ultrasound image, the nodule benign and malignant categories may include benign and malignant categories and TI-RADS grading categories, and the nodule feature categories may include orientation categories, margin categories, acoustic halo categories, structural categories, echo categories, echo texture categories, focal hyperechoic categories, and posterior echo feature categories. As shown in Table 2, when the target image is a breast ultrasound image, the nodule benign and malignant categories may include tissue composition categories, benign and malignant categories, and TI-RADS grading categories, and the nodule feature categories may include morphology categories, orientation categories, margin categories, echo pattern categories, posterior echo feature categories, and calcification categories.

[0078] Table 1 Description of benign and malignant nodule categories and nodule feature categories in thyroid ultrasound images

[0079]

[0080] Table 2 Description of benign and malignant nodule categories and nodule feature categories in breast ultrasound images

[0081]

[0082] In one implementation of this embodiment, the preset detection network model includes a multi-attribute detection module, and determining the benign or malignant category of the nodule and the nodule feature category corresponding to the target image based on the plurality of weighted feature maps specifically includes:

[0083] The weighted feature maps are input into a preset multi-attribute detection module, and the sub-attribute features corresponding to the target image are determined by the preset multi-attribute detection module. The benign or malignant category of the nodule and the nodule feature category corresponding to the target image are determined based on the sub-attribute features.

[0084] Specifically, the number of the sub-attribute features is the same as the number of benign and malignant nodule categories and the number of classification categories included in the nodule feature category, and the sub-attribute features have a one-to-one correspondence with the benign and malignant nodule categories and the classification categories included in the nodule feature category, and each sub-attribute feature is used to determine its corresponding classification category. Figure 2 As shown, the multi-attribute detection module may include several parallel detection units, each of which is connected to a fusion module. The fusion module includes several fusion units, which correspond to and are connected to several detection units one by one. The fusion unit inputs the weighted feature map output by it into its corresponding detection unit, so that the detection unit determines the sub-attribute feature based on the weighted feature map, and determines the predicted category of its corresponding classification category based on the sub-attribute feature.

[0085] In one implementation of this embodiment, the preset detection network model is a trained network model, and the loss function used by the preset detection network model during the training process includes a classification loss term, an attention loss term, and a segmentation loss term, wherein the classification loss term is determined based on the predicted categories and category labels corresponding to the training image determined by the multi-attribute classification module, the attention loss term is determined based on the candidate categories and category labels corresponding to the training image determined by the attention module, and the segmentation loss term is determined based on the contour feature map and the contour mask map. The classification task, activation task, and segmentation task of each attribute of the nodule in this embodiment are interrelated, so the classification loss, attention loss, and nodule segmentation loss are fused to form a multi-task mechanism that can promote each other, thereby improving the model performance of the trained preset detection network model.

[0086] The attention loss term is used to drive the attention module to generate a more accurate activation map to improve the model performance of the preset detection model, wherein the attention loss term L att The calculation formula can be:

[0087]

[0088] Among them, C represents the number of classification categories included in the benign and malignant categories of nodules and the nodule feature categories, represents the candidate category corresponding to the training image determined based on the attention module, l c is the category label and I represents the image of interest.

[0089] The segmentation loss term is used to enable the preset detection network model to learn the accurate nodule contour, where the segmentation loss term L Dice The calculation formula can be:

[0090]

[0091] Among them, N is the number of pixels in the contour feature map, p i Represents the predicted pixel category of the pixel in the contour feature map, g i Represents the pixel class of the pixel in the contour mask

[0092] The classification loss term can use the cross entropy loss function. Correspondingly, the classification loss term L per The calculation formula can be:

[0093]

[0094] Among them, C represents the number of classification categories included in the benign and malignant categories of nodules and the nodule feature categories, represents the predicted category corresponding to the training image determined by the multi-attribute classification module, l cis the category label and I represents the image of interest.

[0095] Based on this, the loss function L of the prediction detection network model can be expressed as:

[0096] L=w per L per +w att L att +w Dice L Dice

[0097] Among them, W per , W att and W Dice Both represent coefficient weights.

[0098] S40: Determine the nodule location and size attributes of the target image according to the contour feature map.

[0099] Specifically, the position attribute is used to reflect the position of the nodule in the target image, and the size attribute is used to reflect the size of the nodule. The size attribute includes the length of the major diameter and the length of the minor diameter. The major diameter refers to the longest line segment in the nodule, and the minor diameter refers to the line segment perpendicular to the major diameter. In addition, after obtaining the size attributes, the major diameter and minor diameter, as well as the length of the major diameter and minor diameter, are annotated in the contour feature map.

[0100] In an implementation of this embodiment, determining the nodule location and size attributes of the target image according to the contour feature map specifically includes:

[0101] Determining the regional position, the length of the major axis, and the length of the minor axis of the nodule region according to the contour feature map, wherein the major axis and the minor axis are perpendicular to each other;

[0102] The region position, the major axis length, and the minor axis length are used as nodule position and size attributes of the target image.

[0103] Specifically, before determining the length of the major diameter and the minor diameter, it is necessary to first determine the major diameter and the minor diameter of the nodule area, and then obtain the major diameter and the minor diameter length by measurement. The process of determining the major diameter and the minor diameter may include: first finding the minimum bounding box of the nodule, that is, a rectangle that can just completely contain the nodule; secondly, determining the two tangent points of the length direction of the rectangle and the nodule, and connecting the two tangent points to obtain the major diameter; finally, determining the perpendicular bisector of the major diameter and obtaining the two intersection points of the perpendicular bisector with the nodule edge, and connecting the two intersection points to obtain the minor diameter.

[0104] In another implementation of this embodiment, the process of determining the major diameter and minor diameter can also be as follows: first, use the principal component analysis (PCA) method to find the main direction and midpoint of the nodule, second, draw a straight line along the main direction and passing through the midpoint, select the line segment formed by the intersection of the straight line and the nodule to obtain the major diameter, and finally, determine the perpendicular bisector of the major diameter and obtain the two intersection points of the perpendicular bisector and the edge of the nodule, and connect the two intersection points to obtain the minor diameter. Of course, in actual use, a combination of the two methods can also be used, for example, first use two methods to determine the major diameter separately, when the difference between the two major diameters (for example, the difference is the difference in the angle between the two major diameters relative to the horizontal direction) is less than a preset threshold (for example, 10 degrees), then randomly select one of the two major diameters as the major diameter of the nodule area, when the difference between the two major diameters is greater than or equal to the preset threshold, the line segment located between the two major diameters can be selected as the major diameter of the nodule area to obtain the major diameter of the nodule area.

[0105] In summary, this embodiment provides a method for detecting thyroid cancer or breast cancer. This method can quickly extract the region of interest of a complete ultrasound image by automatically extracting the region of interest, and can improve the accuracy of region of interest extraction, thereby increasing the effective information carried by the image of interest used to determine the multi-attribute category and reducing invalid information, thereby improving the accuracy of candidate determination of the benign and malignant nodule category, nodule feature category, and attribute information. At the same time, this embodiment provides a preset detection network model, which realizes the fully automatic output from image to classification result and the synchronous acquisition of the benign and malignant nodule category, nodule feature category, and attribute information through the predictive detection network model, thereby improving the detection speed of thyroid cancer or breast cancer, and the preset detection network model can be trained end-to-end. In addition, the nodule feature category and attribute information are conducive to the classification of the benign and malignant nodule category, thereby improving the reliability of the detection results of thyroid cancer or breast cancer.

[0106] Based on the above-mentioned thyroid cancer or breast cancer detection method, this embodiment also provides a thyroid cancer or breast cancer detection device, such as Figure 7 As shown, the device includes:

[0107] An acquisition module 100 is configured to acquire a region of interest of a target image to obtain an image of interest, wherein the target image includes a thyroid image or a breast image, and the target image includes at least one nodule region;

[0108] A feature determination module 200 is configured to determine a plurality of weighted feature maps and a contour feature map of the target image based on the image of interest;

[0109] A classification module 300 is configured to determine a benign or malignant nodule category and a nodule feature category corresponding to the target image based on the weighted feature maps;

[0110] The attribute determination module 400 is configured to determine the nodule location and size attributes of the target image according to the contour feature map.

[0111] Based on the above-mentioned method for detecting thyroid cancer or breast cancer, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the method for detecting thyroid cancer or breast cancer as described in the above-mentioned embodiment.

[0112] Based on the above-mentioned thyroid cancer or breast cancer detection method, the present application also provides a terminal device, such as Figure 8 As shown, it includes at least one processor 20; a display screen 21; and a memory 22. It may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via bus 24. The display screen 21 is configured to display a preset user guidance interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0113] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0114] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0115] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal device. In addition, the memory 22 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0116] In addition, the specific process of loading and executing the multiple instructions in the storage medium and the processor in the terminal device has been described in detail in the above method and will not be described here one by one.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting thyroid cancer or breast cancer, characterized in that: The method comprises: Acquiring a region of interest of a target image to obtain an image of interest, wherein the target image includes a thyroid ultrasound image carrying a thyroid nodule region or a breast ultrasound image carrying a breast nodule region, and the region of interest is an image region with an anatomical structure of a lesion, and the target image includes at least one nodule region; Determining a plurality of weighted feature maps and contour feature maps of the target image according to the image of interest; Determining the benign or malignant category of the nodule and the nodule feature category corresponding to the target image according to the plurality of weighted feature maps; Determining the nodule location and size attributes of the target image according to the contour feature map; wherein the plurality of weighted feature maps and contour feature maps are determined by a preset detection network model, wherein the preset detection network model includes a feature extraction module, an attention module configured with an attention mechanism, a fusion module, and a multi-attribute classification module, the input item of the attention module includes the output item of the feature extraction module, the input item of the fusion module includes the output item of the feature extraction module and the output item of the attention module, and the loss function adopted in the training process of the preset detection network model includes a classification loss item, an attention loss item, and a segmentation loss item, wherein the classification loss item is determined based on each predicted category and each category label corresponding to the training image determined by the multi-attribute classification module, the attention loss item is determined based on the candidate category and each category label corresponding to the training image determined by the attention module, and the segmentation loss item is determined based on the contour feature map and the contour mask map; Furthermore, determining several weighted feature maps and contour feature maps of the target image according to the image of interest includes: Inputting the image of interest into a feature extraction module, and outputting a feature map through the feature extraction module; The feature map is input into an attention module, and the attention module outputs a plurality of activation maps and contour feature maps; wherein each activation map is used to reflect the attention degree of its corresponding classification category to each feature point in the feature map; The activation maps and the feature maps are input into a fusion module, and the fusion module outputs a plurality of weighted feature maps.

2. The method for detecting thyroid cancer or breast cancer according to claim 1, wherein: Determining the benign or malignant category of the nodule and the nodule feature category corresponding to the target image according to the plurality of weighted feature maps specifically includes: The weighted feature maps are input into a multi-attribute classification module, and the multi-attribute classification module determines the sub-attribute features corresponding to the target image, and determines the benign or malignant nodule category and the nodule feature category corresponding to the target image based on the sub-attribute features.

3. The method for detecting thyroid cancer or breast cancer according to claim 1, wherein: The step of determining the nodule location and size attributes of the target image according to the contour feature map specifically includes: Determining the regional position, the length of the major axis, and the length of the minor axis of the nodule region according to the contour feature map, wherein the major axis and the minor axis are perpendicular to each other; The region position, major axis length, and minor axis length are used as nodule position and size attributes of the target image.

4. The method for detecting thyroid cancer or breast cancer according to claim 1, wherein: The acquiring of the region of interest of the target image to obtain the image of interest specifically includes: Converting the target image into a grayscale image, and converting the grayscale image into a binary image; The maximum connected region of the binary image is obtained, and a region of interest of the target image is determined based on the maximum connected region and the target image to obtain an image of interest.

5. A device for detecting thyroid cancer or breast cancer, characterized in that: The device comprises: an acquisition module, configured to acquire a region of interest of a target image to obtain an image of interest, wherein the target image includes a thyroid ultrasound image carrying a thyroid nodule region or a breast ultrasound image carrying a breast nodule region, and the region of interest is an image region with an anatomical structure of a lesion, and the target image includes at least one nodule region; A feature determination module, configured to determine a plurality of weighted feature maps and a contour feature map of the target image based on the image of interest; a classification module, configured to determine, based on the weighted feature maps, a benign or malignant nodule category and a nodule feature category corresponding to the target image; an attribute determination module, configured to determine the nodule position and size attributes of the target image according to the contour feature map; wherein the plurality of weighted feature maps and contour feature maps are determined by a preset detection network model, wherein the preset detection network model includes a feature extraction module, an attention module configured with an attention mechanism, a fusion module, and a multi-attribute classification module, the input item of the attention module includes the output item of the feature extraction module, the input item of the fusion module includes the output item of the feature extraction module and the output item of the attention module, and the loss function adopted in the training process of the preset detection network model includes a classification loss item, an attention loss item, and a segmentation loss item, wherein the classification loss item is determined based on each predicted category and each category label corresponding to the training image determined by the multi-attribute classification module, the attention loss item is determined based on the candidate category and each category label corresponding to the training image determined by the attention module, and the segmentation loss item is determined based on the contour feature map and the contour mask map; Furthermore, the feature determination module is specifically configured to: Inputting the image of interest into a feature extraction module, and outputting a feature map through the feature extraction module; The feature map is input into an attention module, and the attention module outputs a plurality of activation maps and contour feature maps; wherein each activation map is used to reflect the attention degree of its corresponding classification category to each feature point in the feature map; The activation maps and the feature maps are input into a fusion module, and the fusion module outputs a plurality of weighted feature maps.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the thyroid cancer or breast cancer detection method according to any one of claims 1 to 4.

7. A terminal device, characterized in that: include: processor, memory, and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps of the method for detecting thyroid cancer or breast cancer according to any one of claims 1 to 4 are implemented.

Images