A method, device, electronic device and storage medium for detecting breast ultrasound nodules

Abstract

The present invention provides a method, device, electronic device and computer-readable storage medium for detecting breast ultrasound nodules. The method for detecting breast ultrasound nodules provided by the present invention comprises: obtaining an original breast ultrasound image; inputting the original breast ultrasound image into a fully trained target detection network to obtain a tissue distribution prediction heat map, wherein the training process of the target detection network is to input the original breast ultrasound image into the target detection network, and use a loss function to supervise the learning of the target detection network to obtain a fully trained target detection network; and fuse the original breast ultrasound image with the tissue distribution prediction heat map, and input the fused image into the fully trained target detection network to obtain a breast nodule detection result. The method for detecting breast ultrasound nodules provided by the present invention improves the detection accuracy of breast nodules.

Description

Technical Field

[0001] The present invention relates to the technical field of breast nodule detection, and in particular to a breast ultrasonic nodule detection method, device, electronic equipment and computer-readable storage medium. Background Art

[0002] Breast nodules are a symptom that is common in breast hyperplasia (which can form breast cysts) and breast tumor diseases, including benign breast tumors (such as breast fibroids, phyllodes tumors, etc.) and malignant breast tumors (breast cancer). Breast nodules have a high incidence rate. Ultrasound is an important means of detecting breast nodules, but it is currently time-consuming to rely on doctors for examination. Due to the different qualifications of doctors, missed diagnoses and misdiagnoses may occur, such as misjudging ducts as nodules. Although computer-aided diagnosis is a new technology that can improve the efficiency of film reading, the current algorithms for breast ultrasound detection pay little attention to detection accuracy, and the test results may show false positives that violate common sense of anatomical structures, for example, false positives are detected in non-breast areas (such as fat or even muscle layers). Summary of the invention

[0003] The object of the present invention is to provide a method, device, electronic device and computer-readable storage medium for detecting breast nodules by ultrasound, so as to solve the technical problem of low accuracy of breast nodule detection in the prior art.

[0004] The technical solution of the present invention is as follows, which provides a method for detecting breast nodules by ultrasound, comprising the following steps:

[0005] Acquire original breast ultrasound images;

[0006] Inputting the original breast ultrasound image into a fully trained target detection network to obtain a tissue distribution prediction heat map, wherein the training process of the target detection network is to input the original breast ultrasound image into the target detection network, use a loss function to supervise the learning of the target detection network, and obtain a fully trained target detection network;

[0007] The original breast ultrasound image is fused with the tissue distribution prediction heat map, and the fused image is input into the fully trained target detection network to obtain a breast nodule detection result.

[0008] Furthermore, the original breast ultrasound image is input into the target detection network, and the target detection network learning is supervised by using a loss function, including:

[0009] Inputting the original breast ultrasound image into the target detection network, using a loss function to supervise the learning of the target detection network, and outputting a tissue distribution prediction heat map;

[0010] The current tissue distribution prediction heat map is fused with the original breast ultrasound image, the fused image is input into the target detection network, the target detection network learning is supervised by the loss function, the tissue distribution prediction heat map is re-output, and the above steps are repeated until the preset number of executions is reached.

[0011] Furthermore, the structure of the target detection network includes a backbone, a feature pyramid network, a convolutional layer segmentation head and a pointhead, the backbone, the feature pyramid network, and the convolutional layer segmentation head are connected in sequence, and the feature pyramid network is also connected to the pointhead.

[0012] Furthermore, the breast ultrasound nodule detection method also includes, after the original breast ultrasound image is input into the target detection network, the backbone and feature pyramid network extract multi-scale features in the original breast ultrasound image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, and the pointhead outputs the breast nodule detection result.

[0013] Furthermore, the use of loss functions to supervise the learning of the target detection network includes: using a Dice loss function to constrain the tissue distribution prediction heat map, using an IoU loss function and a cross entropy loss function to constrain the breast nodule detection results, and using a focalloss loss function to constrain the separation of the foreground and background of the target detection network.

[0014] Furthermore, the calculation formula of the Dice loss function is: The calculation formula of the IoU loss function is: The calculation formula of the focal loss function is CE(p t )=-α t log(p t ), Among them, DiceLoss is the Dice loss, y true The gold standard for tissue distribution maps, y pred is the tissue distribution map predicted by the network, pixels is the pixel, IoULoss is the IoU loss, A is the labeled breast nodule border, B is the breast nodule border predicted by the network, CE(p t ) is the focal loss, α t is the weight, p is the foreground and background probability map, and y is the probability that the image is the foreground.

[0015] Furthermore, the training process of the target detection network also includes using a multi-scale training strategy for the original breast ultrasound image input into the target detection network, and adjusting the learning rate of the target detection network through a cosine annealing learning rate strategy with warmup.

[0016] Another technical solution of the present invention is as follows: a breast ultrasound nodule detection device is provided, comprising an image acquisition module, a thermal map acquisition module and a detection module;

[0017] The image acquisition module is used to acquire original breast ultrasound images;

[0018] The heat map acquisition module is used to input the original breast ultrasound image into a fully trained target detection network to obtain a tissue distribution prediction heat map, wherein the training process of the target detection network is to input the original breast ultrasound image into the target detection network, use a loss function to supervise the learning of the target detection network, and obtain a fully trained target detection network;

[0019] The detection module is used to fuse the original breast ultrasound image with the tissue distribution prediction heat map, and input the fused image into the fully trained target detection network to obtain a breast nodule detection result.

[0020] Another technical solution of the present invention is as follows: an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the breast ultrasound nodule detection method as described in the above technical solution is implemented.

[0021] Another technical solution of the present invention is as follows: a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and wherein when the computer program is executed by a processor, the method for detecting breast nodules by ultrasound is implemented as described in any of the above technical solutions.

[0022] The beneficial effects of the present invention are: obtaining an original breast ultrasound image, inputting the original breast ultrasound image into a well-trained target detection network, obtaining a tissue distribution prediction heat map, fusing the original breast ultrasound image with the tissue distribution prediction heat map, and inputting the fused image into a well-trained target detection network to obtain a breast nodule detection result; through the above method, the detection accuracy of breast nodules is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the flow chart of the method for detecting breast nodules by ultrasound according to the first embodiment of the present invention;

[0024] Figure 2A schematic diagram of the structure of a target detection network according to a first embodiment of the present invention;

[0025] Figure 3 It is a structural schematic diagram of a breast ultrasound nodule detection device according to a second embodiment of the present invention;

[0026] Figure 4 FIG. 4 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] Figure 1 FIG. 1 is a flow chart of the method for detecting breast nodules by ultrasound according to the first embodiment of the present invention. It should be noted that the method for detecting breast nodules by ultrasound according to the present invention is not limited to the method for detecting breast nodules by ultrasound according to the first embodiment of the present invention if the results are substantially the same. Figure 1 The process sequence shown is limited. Figure 1 As shown, the breast ultrasound nodule detection method mainly includes the following steps:

[0030] S101, obtaining an original breast ultrasound image;

[0031] S102, inputting the original breast ultrasound image into a fully trained target detection network to obtain a tissue distribution prediction heat map, wherein the training process of the target detection network is to input the original breast ultrasound image into the target detection network, use a loss function to supervise the learning of the target detection network, and obtain a fully trained target detection network;

[0032] S103, fusing the original breast ultrasound image with the tissue distribution prediction heat map, and inputting the fused image into the fully trained target detection network to obtain a breast nodule detection result.

[0033] In an optional embodiment, the original breast ultrasound image is input into the target detection network, and the target detection network learning is supervised by a loss function, including:

[0034] Inputting the original breast ultrasound image into the target detection network, using a loss function to supervise the learning of the target detection network, and outputting a tissue distribution prediction heat map;

[0035] The current tissue distribution prediction heat map is fused with the original breast ultrasound image, the fused image is input into the target detection network, the target detection network learning is supervised by the loss function, the tissue distribution prediction heat map is re-output, and the above steps are repeated until the preset number of executions is reached.

[0036] In a specific embodiment, during the training process of the target detection network, the original breast ultrasound image is input into the target detection network, and a tissue distribution prediction heat map is output. The tissue distribution prediction heat map is the first tissue distribution prediction heat map. The first tissue distribution prediction heat map is fused with the original breast ultrasound image at a set ratio. For example, the set ratio may be 4:1. The fused image is then input into the target detection network, and a second tissue distribution prediction heat map is output. The second tissue distribution prediction heat map is fused with the original breast ultrasound image at a set ratio to obtain a third tissue distribution prediction heat map. The above image fusion and input process is repeated until the preset number of executions n is reached. The nth tissue distribution prediction heat map is the breast nodule detection result. The set ratio for each image fusion may be the same or different. Repeated fusion and training of images can accelerate the training of the convolutional layer segmentation head.

[0037] Among them, when formally performing breast nodule detection, it is only necessary to fuse the original breast ultrasound image with the tissue distribution prediction heat map once, input the fused image into the target detection network, and output the breast nodule detection result.

[0038] In an optional embodiment, the structure of the target detection network includes a backbone, a feature pyramid network, a convolutional layer segmentation head and a point head, the backbone, the feature pyramid network, and the convolutional layer segmentation head are connected in sequence, and the feature pyramid network is also connected to the point head.

[0039] In a specific embodiment, the target detection network is an improved reppoints detection network. The reppoints detection network is a one-stage detector with the characteristics of being lightweight and high-speed. The structural diagram of the target detection network is as follows: Figure 2 As shown, Figure 2It includes an image feature extraction part, a tissue probability prior prediction part and a detection part. The image feature extraction part includes a backbone and a feature pyramid network (FPN). The image feature extraction part is used to extract multi-scale features of the image. The tissue probability prior prediction part includes a single-layer convolutional layer segmentation head (segmentation head). The tissue probability prior prediction part is used to convert the multi-scale features of the image extracted by the image feature extraction part into a tissue distribution prediction heat map (tissue probability distribution). The structural block diagram of the segmentation head is as follows: Figure 2 As shown in the largest rectangle in the figure; the detection part includes pointhead, which is used to output the final breast nodule detection result, that is, the bounding box coordinates of the nodule. Figure 2 In the figure, from left to right, the first picture is the original breast ultrasound image at the initial moment, the second picture is the image after the original breast ultrasound image is fused with the tissue distribution prediction heat map, the third picture is the tissue distribution prediction heat map, and the fourth picture is the foreground and background probability map. Figure 2 As shown in the figure, the object detection network is trained for n (multiple) cycles. The images input to the feature pyramid network are images of 5 different sizes and channels, and the feature pyramid network outputs 5 images of different sizes and 256 channels.

[0040] In an optional embodiment, the breast ultrasound nodule detection method also includes, after the original breast ultrasound image is input into the target detection network, the backbone and feature pyramid network extract multi-scale features in the original breast ultrasound image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, and the point head outputs the breast nodule detection result.

[0041] It should be noted that the convolutional layer segmentation head includes up and down sampling layers and a convolutional layer, whose input is the multi-scale feature map output by the feature pyramid network, and its output is the tissue distribution prediction heat map with the same size as the input image.

[0042] Among them, in the embodiment of the present invention, the tissue distribution of the breast is used as prior knowledge, which can effectively avoid detecting nodules in non-breast tissues and improve the interpretability of the network. The expression of the tissue distribution is expressed in the form of semantic segmentation. Specifically, pixels belonging to the same tissue are marked with the same value, and a total of 3 types of tissues are marked, namely fat and skin, breast and nodules, and posterior space and muscle tissue. The final marking form is a 3-channel image, and each channel is a binary image. The number of channels of the above-mentioned tissue distribution prediction heat map is 3, and the 3 channel semantics of the tissue distribution prediction heat map correspond to the tissue distribution labels of the original breast ultrasound image, and the 3 channel semantics are fat and skin, breast and nodules, and posterior space and muscle tissue.

[0043] In a specific embodiment, when training the target detection network, after the original breast ultrasound image is input into the target detection network, the backbone and feature pyramid network extract multi-scale features in the original breast ultrasound image, and the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map; the tissue distribution prediction heat map is fused with the original breast ultrasound image to obtain a fused image, the backbone and feature pyramid network extract multi-scale features in the fused image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, and the current tissue distribution prediction heat map is fused with the original breast ultrasound image. The above steps are performed until the preset number of executions is reached, and the pointhead outputs the breast nodule detection result. When the breast nodules are formally detected, the tissue distribution prediction heat map is fused with the original breast ultrasound image once, and then re-input into the target detection network, and then the detection result is output. The output detection result includes the category and location of the tissue distribution.

[0044] In an optional embodiment, the use of a loss function to supervise the learning of the target detection network includes: using a Dice loss function to constrain the tissue distribution prediction heat map, using an IoU loss function and a cross entropy loss function to constrain the breast nodule detection results, and using a focalloss loss function to constrain the separation of the foreground and background of the target detection network.

[0045] In an optional embodiment, the calculation formula of the Dice loss function is: The calculation formula of the IoU loss function is: The calculation formula of the focal loss function is CE(p t )=-α t log(p t ), Among them, DiceLoss is the Dice loss, y true The gold standard for tissue distribution maps, y pred is the tissue distribution map predicted by the network, pixels is the pixel, IoULoss is the IoU loss, A is the labeled breast nodule border, B is the breast nodule border predicted by the network, CE(p t ) is the focal loss, α t is the weight, p is the foreground and background probability map, and y is the probability that the image is the foreground.

[0046] In a specific embodiment, the Dice loss function is used to constrain the tissue distribution prediction heat map, and then the IoU loss function and the cross entropy loss function are used to constrain the final detection result. The focal loss loss function is used to constrain the foreground and background separation of the target detection network. The gold standard of the tissue distribution map is the correct tissue distribution annotated by the doctor, and the annotated breast nodule bounding box is the breast nodule bounding box annotated by the doctor. t It can be set to 1.5. When it is 1, the focalloss loss function degenerates into the ordinary cross entropy loss function.

[0047] In an optional embodiment, the training process of the target detection network further includes using a multi-scale training strategy for the original breast ultrasound image input to the target detection network, and adjusting the learning rate of the target detection network through a cosine annealing learning rate strategy with warmup.

[0048] Among them, a multi-scale training strategy is used to ensure that the target detection network is robust to scale changes. The multi-scale training strategy is specifically set as the input image is randomly scaled to 0.5 to 2 times the size of the original image. A cosine annealing learning rate strategy with warmup is used to ensure that the training process is stable and converges quickly. The specific settings of the cosine annealing learning rate strategy with warmup are warmup 5 epochs, cosine annealing 200 epochs, maximum learning rate 0.001, and minimum learning rate 0.0000001.

[0049] In a specific embodiment, the training of the target detection network can use quantitative and qualitative analysis as performance evaluation indicators of the target detection network. The quantitative analysis uses commonly used indicators for target detection, namely AP, Recall and Precision. In the qualitative analysis, according to the target detection network in the embodiment of the present invention, that is, the improved reppoints detection network and the baseline model (that is, the original reppoints detection network) The error detection situation in the non-breast tissue part determines whether the target detection network in the embodiment of the present invention will suppress the misdetection of the non-breast tissue part after introducing the tissue distribution prior.

[0050] In another specific embodiment, a method for detecting breast ultrasound nodules is provided, 625 breast ultrasound images are obtained; the target detection network is trained, and the trained target detection network is tested using 830 validation sets and 820 test sets, which are not repeated with the validation set, the test set and the training set; the original breast ultrasound image is input into the fully trained target detection network to obtain a tissue distribution prediction heat map; the original breast ultrasound image is fused with the tissue distribution prediction heat map, and the fused image is input into the fully trained target detection network to obtain a breast nodule detection result. The original breast ultrasound image is detected using the baseline model to obtain a detection result, and an indicator comparison table of the corresponding detection results of the two methods is obtained, as shown in Table 1.

[0051] Table 1 Comparison of indicators of the corresponding detection results of the two methods

[0052]

[0053] Among them, the detection method of the original Reppoints network is a method for detection using the existing Reppoints network. According to Table 1, the breast ultrasound nodule detection method provided by the embodiment of the present invention can improve the index AP from 0.797 to 0.878, and it has the advantage of low missed detection rate. In addition, the breast ultrasound nodule detection method provided by the embodiment of the present invention first predicts different tissue structures, and then further uses the predicted tissue structures to complete nodule detection, so that the breast ultrasound nodule detection method provided by the present invention has good interpretability; the breast ultrasound nodule detection method provided by the embodiment of the present invention improves the precision from 0.816 to 0.850, and there is almost no misdetection in non-breast areas (such as muscle areas, behind the ribs); it can be seen that the breast ultrasound nodule detection method meets the actual clinical use needs.

[0054] The breast ultrasound nodule detection method provided by the embodiment of the present invention obtains the original breast ultrasound image, inputs the original breast ultrasound image into a fully trained target detection network, obtains a tissue distribution prediction heat map, fuses the original breast ultrasound image with the tissue distribution prediction heat map, and inputs the fused image into the fully trained target detection network to obtain a breast nodule detection result; through the above method, the detection accuracy of breast nodules is improved. The fully trained target detection network provided by the embodiment of the present invention only needs to input an ultrasound image, and on the basis of generating a tissue distribution map, reuses the generated tissue distribution map to synthesize a new input image and detect whether a nodule exists at the corresponding position. The target detection network provided by the embodiment of the present invention realizes multi-task learning by adding a segmentation head to predict tissue distribution, and reuses the prediction result after predicting the tissue distribution and fuses it with the original input, and finally outputs the detection result.

[0055] The breast ultrasound nodule detection method provided in this embodiment can be constructed based on artificial intelligence, and relevant data can be acquired and processed based on artificial intelligence technology to achieve unattended artificial intelligence breast ultrasound nodule detection. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0056] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0057] Figure 3 FIG. 2 is a schematic diagram of the structure of a breast ultrasound nodule detection device according to a second embodiment of the present invention. Figure 3 As shown, the breast ultrasound nodule detection device 20 includes an image acquisition module 21, a heat map acquisition module 22 and a detection module 23; the image acquisition module 21 is used to acquire an original breast ultrasound image; the heat map acquisition module 22 is used to input the original breast ultrasound image into a fully trained target detection network to acquire a tissue distribution prediction heat map, wherein the training process of the target detection network is to input the original breast ultrasound image into the target detection network, use a loss function to supervise the learning of the target detection network, and obtain a fully trained target detection network; the detection module 23 is used to fuse the original breast ultrasound image with the tissue distribution prediction heat map, and input the fused image into the fully trained target detection network to obtain a breast nodule detection result.

[0058] Furthermore, the breast ultrasound nodule detection device 20 also includes a model training module 21, which is used to input the original breast ultrasound image into the target detection network, supervise the learning of the target detection network using a loss function, and output a tissue distribution prediction heat map; fuse the current tissue distribution prediction heat map with the original breast ultrasound image, input the fused image into the target detection network, supervise the learning of the target detection network using a loss function, re-output the tissue distribution prediction heat map, and repeat the above steps until a preset number of executions is reached.

[0059] Wherein, during the training process of the target detection network, the original breast ultrasound image is input into the target detection network, and a tissue distribution prediction heat map is output, which is a first tissue distribution prediction heat map. The first tissue distribution prediction heat map is fused with the original breast ultrasound image according to a set ratio, for example, the set ratio can be 4:1, and then the fused image is input into the target detection network, and a second tissue distribution prediction heat map is output. The second tissue distribution prediction heat map is fused with the original breast ultrasound image according to a set ratio to obtain a third tissue distribution prediction heat map. The above image fusion and input process is repeated until the preset number of executions n is reached, and the nth tissue distribution prediction heat map is the breast nodule detection result. Wherein, the set ratio of each image fusion can be the same or different. Repeated fusion and training of images can accelerate the training of the convolutional layer segmentation head.

[0060] Among them, the structure of the target detection network includes a backbone, a feature pyramid network, a convolutional layer segmentation head and a pointhead, the backbone, the feature pyramid network, and the convolutional layer segmentation head are connected in sequence, and the feature pyramid network is also connected to the point head.

[0061] The target detection network is an improved reppoints detection network, which is a one-stage detector with the characteristics of light weight and high speed. The convolutional layer segmentation head includes up and down sampling layers and a convolutional layer. Its input is a multi-scale feature map output by the feature pyramid network, and its output is a tissue distribution prediction heat map with the same size as the input image. The tissue distribution of the breast can be used as prior knowledge, which can well avoid detecting nodules in non-breast tissues and improve the interpretability of the network. The expression of semantic segmentation is used as the expression form of tissue distribution. Specifically, pixels belonging to the same tissue are marked with the same value. A total of three types of tissues are marked, namely fat and skin, breast and nodules, and posterior gap and muscle tissue. The final marking form is a 3-channel image, and each channel is a binary image. The number of channels of the above tissue distribution prediction heat map is 3. The three channel semantics of the tissue distribution prediction heat map correspond to the tissue distribution labels of the original breast ultrasound image, and the three channel semantics are fat and skin, breast and nodules, and posterior gap and muscle tissue.

[0062] Wherein, after the original breast ultrasound image is input into the target detection network, the backbone and feature pyramid network extract the multi-scale features in the original breast ultrasound image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, and the point head outputs the breast nodule detection result. When training the target detection network, after the original breast ultrasound image is input into the target detection network, the backbone and feature pyramid network extract the multi-scale features in the original breast ultrasound image, and the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map; the tissue distribution prediction heat map is fused with the original breast ultrasound image to obtain a fused image, the backbone and feature pyramid network extract the multi-scale features in the fused image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, the current tissue distribution prediction heat map is fused with the original breast ultrasound image, the above steps are performed until the preset number of executions is reached, and the point head outputs the breast nodule detection result. When formally detecting breast nodules, the tissue distribution prediction heat map is fused with the original breast ultrasound image once, and then re-input into the target detection network to output the detection result. The output detection result includes the type and location of the tissue distribution.

[0063] Furthermore, the model training module 21 is also used to use the Dice loss function to constrain the tissue distribution prediction heat map, use the IoU loss function and the cross entropy loss function to constrain the breast nodule detection results, and use the focalloss loss function to constrain the separation of the foreground and background of the target detection network.

[0064] Among them, the calculation formula of the Dice loss function is: The calculation formula of the IoU loss function is: The calculation formula of the focal loss function is CE(p t )=-α t log(p t ), Among them, DiceLoss is the Dice loss, y true The gold standard for tissue distribution maps, y pred is the tissue distribution map predicted by the network, pixels is the pixel, IoULoss is the IoU loss, A is the labeled breast nodule border, B is the breast nodule border predicted by the network, CE(p t ) is the focal loss, α t is the weight, p is the foreground and background probability map, and y is the probability that the image is the foreground.

[0065] In a specific embodiment, the Dice loss function is used to constrain the tissue distribution prediction heat map, and then the IoU loss function and the cross entropy loss function are used to constrain the final detection result. The focalloss loss function is used to constrain the foreground and background separation of the target detection network. The gold standard of the tissue distribution map is the correct tissue distribution annotated by the doctor, and the annotated breast nodule border is the breast nodule border annotated by the doctor. Among them, the weight α t It can be set to 1.5. When it is 1, the focalloss loss function degenerates into the ordinary cross entropy loss function.

[0066] Among them, a multi-scale training strategy is used for the original breast ultrasound image input into the target detection network, and the learning rate of the target detection network is adjusted by a cosine annealing learning rate strategy with warmup.

[0067] Among them, a multi-scale training strategy is used to ensure that the target detection network is robust to scale changes. The multi-scale training strategy is specifically set as the input image is randomly scaled to 0.5 to 2 times the size of the original image. A cosine annealing learning rate strategy with warmup is used to ensure that the training process is stable and converges quickly. The specific settings of the cosine annealing learning rate strategy with warmup are warmup 5 epochs, cosine annealing 200 epochs, maximum learning rate 0.001, and minimum learning rate 0.0000001.

[0068] In a specific embodiment, the training of the target detection network can use quantitative and qualitative analysis as performance evaluation indicators of the target detection network. The quantitative analysis uses commonly used indicators for target detection, namely AP, Recall and Precision. In the qualitative analysis, according to the target detection network in the embodiment of the present invention, that is, the improved reppoints detection network and the baseline model, the error detection situation in the non-breast tissue part is judged whether the target detection network in the embodiment of the present invention will suppress the false detection of the non-breast tissue part after introducing the tissue distribution prior.

[0069] Figure 4 Schematic diagram of the structure of the electronic device according to the third embodiment of the present invention. Figure 4 As shown, the electronic device 30 includes a processor 31 and a memory 32 that is communicatively connected to the processor 31 .

[0070] The memory 32 stores program instructions for implementing the breast ultrasound nodule detection method of any of the above embodiments.

[0071] The processor 31 is used to execute program instructions stored in the memory 32 to perform code testing.

[0072] The processor 31 may also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip having signal processing capabilities. The processor 31 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0073] The memory 32 may be used to store the computer programs and / or modules, and the processor 31 implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory 32, and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, and the like.

[0074] The memory 32 may be integrated into the processor 31 , or may be provided separately from the processor 31 .

[0075] The fourth embodiment of the present invention provides a storage medium, the storage medium of the embodiment of the present invention stores program instructions that can implement all the above methods, and the storage medium can be non-volatile or volatile. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM). Among them, the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or CD and other media that can store program code, or terminal devices such as computers, servers, mobile phones, tablets, etc.

[0076] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0077] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present invention.

[0078] The above description is only an implementation mode of the present invention. It should be pointed out that, for ordinary technicians in this field, improvements can be made without departing from the creative concept of the present invention, but these all belong to the protection scope of the present invention.

Claims

1. A method for detecting breast nodules by ultrasound, characterized in that: The steps include: Acquire original breast ultrasound images; Input the original breast ultrasound image into a fully trained target detection network to obtain a tissue distribution prediction heat map, wherein the structure of the target detection network includes a backbone, a feature pyramid network, a convolutional layer segmentation head and a point head, the backbone, the feature pyramid network and the convolutional layer segmentation head are connected in sequence, the feature pyramid network is also connected to the point head, the training process of the target detection network is as follows: input the original breast ultrasound image into the target detection network, the backbone and the feature pyramid network extract multi-scale features in the original breast ultrasound image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, the point head outputs a breast nodule detection result, and the target detection network learning is supervised by a loss function to obtain a fully trained target detection network; The original breast ultrasound image is fused with the tissue distribution prediction heat map, and the fused image is input into the fully trained target detection network to obtain a breast nodule detection result.

2. The method for detecting breast nodules by ultrasound according to claim 1, characterized in that: The using a loss function to supervise the target detection network learning includes: Using a loss function to supervise the learning of the target detection network and output a tissue distribution prediction heat map; The current tissue distribution prediction heat map is fused with the original breast ultrasound image, the fused image is input into the target detection network, the target detection network learning is supervised by the loss function, the tissue distribution prediction heat map is re-output, and the above steps are repeated until the preset number of executions is reached.

3. The method for detecting breast nodules by ultrasound according to claim 1, characterized in that: The use of loss functions to supervise the learning of the target detection network includes: using a Dice loss function to constrain the tissue distribution prediction heat map, using an IoU loss function and a cross entropy loss function to constrain the breast nodule detection result, and using a focalloss loss function to constrain the separation of the foreground and background of the target detection network.

4. The method for detecting breast nodules by ultrasound according to claim 3, characterized in that: The calculation formula of the Dice loss function is: The calculation formula of the IoU loss function is: The calculation formula of the focal loss function is CE(p t )=-α t log(p t ), Among them, DiceLoss is the Dice loss, y true The gold standard for tissue distribution maps, y pred is the tissue distribution map predicted by the network, pixels is the pixel, IoULoss is the IoU loss, A is the labeled breast nodule border, B is the breast nodule border predicted by the network, CE(p t ) is the focal loss, α t is the weight, p is the foreground and background probability map, and y is the probability that the image is the foreground.

5. The method for detecting breast nodules by ultrasound according to claim 1, characterized in that: The training process of the target detection network also includes using a multi-scale training strategy for the original breast ultrasound image input to the target detection network, and adjusting the learning rate of the target detection network through a cosine annealing learning rate strategy with warmup.

6. A breast ultrasound nodule detection device, characterized in that: It includes an image acquisition module, a heat map acquisition module and a detection module; The image acquisition module is used to acquire original breast ultrasound images; The heat map acquisition module is used to input the original breast ultrasound image into a fully trained target detection network to obtain a tissue distribution prediction heat map, wherein the structure of the target detection network includes a backbone, a feature pyramid network, a convolutional layer segmentation head and a point head, the backbone, the feature pyramid network and the convolutional layer segmentation head are connected in sequence, the feature pyramid network is also connected to the point head, the training process of the target detection network is to input the original breast ultrasound image into the target detection network, the backbone and the feature pyramid network extract multi-scale features in the original breast ultrasound image, the convolutional layer segmentation head converts the multi-scale features into a tissue distribution prediction heat map, the point head outputs a breast nodule detection result, and uses a loss function to supervise the learning of the target detection network to obtain a fully trained target detection network; The detection module is used to fuse the original breast ultrasound image with the tissue distribution prediction heat map, and input the fused image into the fully trained target detection network to obtain a breast nodule detection result.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, wherein: When the processor executes the computer program, the method for detecting breast nodules by ultrasound according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting breast nodules by ultrasound is implemented as described in any one of claims 1 to 5.

Images