A nipple detection method and device, a computer device, and a storage medium

Abstract

An embodiment of the present invention provides a nipple detection method, which includes: obtaining a breast ultrasound image file, automatically aligning the middle position of the coronal plane, and obtaining a coronal plane image of the AP position; preprocessing the image data of the coronal plane image; inputting the preprocessed image into a detection model to obtain reasonable nipple positions and the confidence levels of each reasonable position, where the detection model is obtained by training the yolov5 network based on the training set of the coronal plane image; obtaining the nipple position according to the reasonable nipple positions and the confidence levels. By performing AI nipple detection on three-dimensional breast ultrasound images, it is no longer necessary to manually confirm the nipple position during the scanning process when using three-dimensional breast ultrasound, solving the pain point that manual confirmation of the nipple position is required during current three-dimensional breast ultrasound examinations.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a nipple detection method and device, a computer device, and a storage medium. Background Art

[0002] In today's rapid development of medical technology and medical equipment, various imaging and ultrasound technologies have been increasingly applied to various fields of medical diagnosis. Medical diagnostic CT images or ultrasound technologies related to the breast are also commonly used, such as the diagnosis of breast cancer and various breast plastic and reconstructive surgeries. In the diagnosis related to the breast, the positioning of breast tissue needs to be based on the position of the nipple in a series of pictures.

[0003] For traditional nipple detection methods, on the one hand, the position of the nipple and the breast are judged according to CT images or ultrasound images, usually using manual methods, usually judged manually by doctors. However, manual judgment is time-consuming and laborious, increasing the workload of doctors.

[0004] With the development of computer technology, the automated processing of CT images or ultrasound images has become a trend. Through computer-aided processing, complex and mechanized manual operations can be made convenient and fast. However, there are many other structures near the nipple, which will cause confusion about the position of the nipple. In CT images or ultrasound images, it is difficult for a computer to accurately find the position of the nipple. How to quickly and accurately judge the nipple and related parts in a large number of image images has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a nipple detection method and device, as well as a computer device and a storage medium, so that when volumetric breast ultrasound scans the entire breast tissue, the breast ultrasound images can be automatically detected, and manual confirmation of the nipple position is no longer required during the ultrasound scan.

[0006] To achieve the above-mentioned invention purpose, in a first aspect, an embodiment of the present invention provides a nipple detection method, the method includes: obtaining a breast ultrasound image file, automatically performing alignment at the middle position of the coronal plane, and obtaining a coronal plane image of the AP position;

[0007] Performing preprocessing on the image data of the coronal plane image;

[0008] Inputting the image obtained by preprocessing into a detection model to obtain reasonable nipple positions and the confidence levels of each reasonable position, where the detection model is obtained by training the yolov5 network based on the training set of the coronal plane image;

[0009] Obtaining the nipple position according to the reasonable nipple positions and the confidence levels.

[0010] Preferably, the preprocessing of the image data for the coronal plane image specifically includes:

[0011] Selecting an image in a predetermined depth interval located at the uppermost coronal plane image;

[0012] Intercepting five pictures each in three specified depth intervals from the images in the predetermined depth interval;

[0013] Performing a unified image pixel point spacing operation on the five pictures obtained in each of the three depth intervals;

[0014] Adding the pixel values corresponding to the five pictures in the same depth interval and then taking the average to obtain one coronal plane picture for each of the three depth intervals;

[0015] Putting these three coronal plane pictures into the RGB three channels respectively.

[0016] Preferably, inputting the image obtained by preprocessing into the nipple detection model to obtain the reasonable nipple position and the confidence level of each reasonable position specifically includes:

[0017] The nipple detection model outputs the coordinates of multiple nipple positions, obtaining a point set R1 of the coordinates that fall within the reasonable nipple position interval, or a point set R2 of the coordinates that do not fall within the reasonable nipple position interval;

[0018] At the same time, obtaining the confidence levels of the coordinates in each point set.

[0019] Preferably, obtaining the nipple position according to the reasonable nipple position and the confidence level specifically includes:

[0020] If R1 is not empty, then selecting the coordinate with the maximum confidence level in the point set R1 as the nipple position;

[0021] If R1 is empty, then selecting the coordinate with the maximum confidence level in R2 as the nipple position.

[0022] In a second aspect, an embodiment of the present invention provides a nipple detection device, and the device includes:

[0023] An image acquisition module, configured to acquire a breast ultrasound image file, automatically perform alignment at the middle position of the coronal plane, and acquire a coronal plane image in the AP position;

[0024] An image preprocessing module, configured to perform preprocessing of image data on the coronal plane image;

[0025] A detection module, configured to input the image obtained by preprocessing into the detection model to obtain the reasonable nipple position and the confidence level of each reasonable position, and the detection model is obtained by training the yolov5 network based on the training set of the coronal plane image;

[0026] A nipple position acquisition module, configured to obtain the nipple position according to the reasonable nipple position and the confidence level.

[0027] Preferably, the image preprocessing module specifically includes:

[0028] A selection unit, configured to select an image located in a predetermined depth interval of the coronal plane image closest to the top layer;

[0029] A screenshot unit, configured to intercept five pictures each in three specified depth intervals from the images in the predetermined depth interval;

[0030] An image calculation unit, configured to perform a unified image pixel point spacing (spacing) operation on the five pictures obtained in each of the three depth intervals, and add and average the pixel values corresponding to the five pictures in the same depth interval to obtain one coronal plane picture for each of the three depth intervals, and put these three coronal plane pictures into the RGB three channels respectively.

[0031] Preferably, the detection module specifically includes:

[0032] An output unit, configured to obtain the coordinates of multiple nipple positions output by the nipple detection model, to obtain a point set R1 of the coordinates that fall within the reasonable nipple position interval, or a point set R2 of the coordinates that do not fall within the reasonable nipple position interval;

[0033] A confidence level unit, configured to obtain the confidence level of the coordinates in each point set.

[0034] Preferably, the nipple position acquisition module is specifically configured to:

[0035] If R1 is not empty, select the coordinate with the maximum confidence level in the point set R1 as the nipple position;

[0036] If R1 is empty, select the coordinate with the maximum confidence level in R2 as the nipple position.

[0037] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the nipple detection method as described above.

[0038] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium. At least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the nipple detection method as described above.

[0039] With the above embodiments, the present invention performs AI nipple detection on three-dimensional breast ultrasound images, so that when using three-dimensional breast ultrasound, there is no need for manual confirmation of the nipple position during the scanning process, solving the pain point of the need for manual confirmation of the nipple position during current three-dimensional breast ultrasound examinations, saving manpower, improving the scanning efficiency, and the output results are objective and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The following will illustrate the specific embodiments of the present invention with reference to the drawings.

[0041] Figure 1 It is a schematic diagram of a nipple detection method provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic coronal plane diagram showing three-dimensional breast ultrasound scanning;

[0043] Figure 3 It is a schematic diagram of an image data preprocessing process provided by an embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram of the network structure of the yolov5 algorithm in the nipple automatic detection model provided by an embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of a nipple detection device provided by an embodiment of the present invention;

[0046] Figure 6 It is a structural block diagram of an image preprocessing unit in a nipple detection device provided by an embodiment of the present invention;

[0047] Figure 7 It is a structural block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific embodiments of the present invention will be described below with reference to the drawings. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, and other embodiments can be obtained. To make the drawings concise, only the parts related to the present invention are schematically shown in each figure, and they do not represent the actual structure of the product.

[0049] As Figure 1 shown, an embodiment of the present invention provides a nipple detection method, and the method includes:

[0050] S1. Obtain a breast ultrasound image file, automatically align the middle position of the coronal plane, and obtain the coronal plane image in the AP position;

[0051] Obtain a breast ultrasound image file, where the breast ultrasound image file can be stored in the format of a dcm file, and there is no information about the nipple position in the dcm file;

[0052] Open the breast ultrasound image file, and automatically align the middle position of the coronal plane, that is, align the AP position;

[0053] As Figure 2 shown, it is a schematic diagram of the coronal plane of breast three-dimensional ultrasound scanning. Different from two-dimensional ultrasound, three-dimensional ultrasound images have rich coronal plane information and include relatively clear nipple position information.

[0054] S2. Preprocess the image data of the coronal plane image;

[0055] Preferably, the preprocessing of the image data of the coronal plane image specifically includes: selecting an image in a predetermined depth interval located at the coronal plane image closest to the top layer; intercepting five pictures each in three specified depth intervals in the image in the predetermined depth interval; performing a unified image pixel point spacing (spacing) operation on the five pictures obtained in each of the three depth intervals; and adding the pixel values of the five pictures corresponding to the same depth interval and then taking the average to obtain one coronal plane picture for each of the three depth intervals; putting these three coronal plane pictures into the RGB three channels respectively.

[0056] Since three-dimensional ultrasound images have rich coronal plane information and include relatively clear nipple position information, the embodiments of the present invention first preprocess the image data on the coronal plane; at the same time, since the nipple position is usually displayed in the AP position, the coronal plane image in the AP position is selected.

[0057] As Figure 3 shown, it is a schematic diagram of the image data preprocessing process. In Figure 3 , in the three-dimensional ultrasound image of the embodiments of the present invention, obvious nipple information is usually located in the image in the depth interval of 1-4 mm from the coronal plane image closest to the top layer. However, at the same time, in order to retain the nipple position information at different depths as much as possible, as Figure 3 shown, the embodiments of the present invention perform the following operations on the images in this depth interval:

[0058] First, intercept five pictures each at three specified depths in this depth interval, that is, intercept five consecutive pictures at the first depth, for example, [1-2.2] mm, five consecutive pictures at the second depth, such as [2-3.2] mm, and five consecutive pictures at the third depth, such as [3-4.2] mm, for a total of 15 pictures.

[0059] Next, in the embodiments of the present invention, a unified image pixel point spacing operation is performed on the 5 images obtained in each of these three depth intervals.

[0060] And the pixel values corresponding to the 5 images in the same interval are added and averaged to generate one image. At this time, one processed coronal plane image is obtained for each of the three intervals. Then, these three images are respectively placed into the RGB three channels of the same image.

[0061] Through the above steps, the embodiments of the present invention obtain a preprocessed image and input it into the nipple detection model.

[0062] S3. Input the preprocessed image into the nipple detection model to obtain reasonable nipple positions and the confidence levels of each reasonable position. The detection model is obtained by training the yolov5 network based on the training set of the coronal plane images;

[0063] Preferably, the inputting the preprocessed image into the nipple detection model to obtain reasonable nipple positions and the confidence levels of each reasonable position specifically includes:

[0064] The nipple detection model outputs the coordinates of multiple nipple positions, obtaining a point set R1 of the coordinates that fall within the reasonable nipple position interval, or a point set R2 of the coordinates that do not fall within the reasonable nipple position interval;

[0065] At the same time, the confidence levels of the coordinates in each point set are obtained.

[0066] In the embodiments of the present invention, the preprocessed image is input into the nipple detection model. The nipple detection model is trained using a large amount of data. The training model uses the yolov5 algorithm, and the yolov5 algorithm is an algorithm that uses a deep convolutional neural network to learn features to detect nipples.

[0067] As Figure 4 shown is the network structure diagram of the yolov5 algorithm.

[0068] The network is mainly divided into four parts: the input end, the Backbone (main network), the PANet (used to enhance the diversity and robustness of features), and the output.

[0069] At the input end, Yolov5 uses adaptive computing anchor boxes to provide anchor box calculation for different data set sample distributions; at the same time, it provides adaptive image scaling for the input images, and performs dynamic adaptive image padding for images with different aspect ratios input in practical applications, which can improve the inference speed.

[0070] In the backbone, specifically, it is the focus structure + CSP bottleneck layer. The key of the focus structure is the slicing operation. After slicing the input image, convolution is performed, and finally the feature map is obtained. The CSP module is used to reduce the computational bottleneck and memory cost. And an SPP layer, that is, spatial pyramid pooling, is added before the last convolutional layer + CSP bottleneck layer. Pooling with different sizes is used and then fused to increase the receptive field.

[0071] In the PANet part, a bottom-up logic is introduced, making it easier for low-level information to be transmitted to the top level. Specifically, it first passes through a convolutional layer (Conv1*1), then the same-dimension ones are concatenated, and then passes through the CSP bottleneck layer module;

[0072] In the Output part, there are mainly three outputs, which are feature maps of different sizes, and finally a final output vector with class probabilities, confidence scores, and bounding boxes is generated.

[0073] In addition, four network structure control methods are provided. The four network structure control methods differ in four different network depths and widths, and can be divided into Yolov5s, Yolov5m, Yolov5l, and Yolov5x, with the network depth gradually deepening and widening. As the network deepens, the feature extraction function and fusion ability of the network also deepen. Which network structure to use specifically can be selected by choosing different depth and width parameters in the yaml configuration file. And in the embodiment of the present invention, yolov5s is used, and the lightest model is used for training.

[0074] In the training of the nipple detection model, the embodiment of the present invention selects multiple three-dimensional ultrasound coronal plane image data as training data, that is, selects three-dimensional ultrasound coronal plane images and nipple position data at a specified depth. Obvious nipple information is usually located in the images in the depth interval of 1 - 4 mm from the topmost coronal plane image. The nipple position data comes from the gold standard marked by doctors.

[0075] At the same time, in order to retain the nipple position information at different depths as much as possible, before training the training model, the following operations are performed on the images in this depth interval: First, 5 images are intercepted at each of the specified 3 depths in this depth, that is, 5 consecutive images are intercepted at a depth of [1 - 2.2] mm, 5 consecutive images are intercepted at a depth of [2 - 3.2] mm, and 5 consecutive images are intercepted at a depth of [3 - 4.2] mm, for a total of 15 images;

[0076] Perform a unified image pixel point spacing operation on the 5 images obtained in these three depth intervals, add the pixel values corresponding to the 5 images in the same interval and then take the average to generate one image, and obtain 1 processed coronal plane image for each of the three intervals;

[0077] Input these three coronal plane images into the three RGB channels respectively.

[0078] Use the prepared images and the nipple position data together as the training set to train the yolov5s model, and then use the test set to test the trained model. The nipple position data comes from the gold standard marked by doctors.

[0079] The image passing through the nipple detection model will output the reasonable nipple position and the confidence level at that position.

[0080] S4. Obtain the nipple position according to the reasonable nipple position and the confidence level.

[0081] When multiple reasonable nipple positions are detected, preferentially select the point with the highest confidence level as the nipple position; when no reasonable nipple position is detected, select the point with the highest confidence level as the nipple position.

[0082] Preferably, the obtaining the nipple position according to the reasonable nipple position and the confidence level specifically includes:

[0083] If R1 is not empty, select the coordinate corresponding to the maximum confidence level in the R1 point set as the nipple position;

[0084] If R1 is empty, select the coordinate corresponding to the maximum confidence level in R2 as the nipple position.

[0085] The nipple detection model outputs the coordinates of multiple nipple positions, and the output result includes the point set R1 of the coordinates falling within the reasonable nipple position interval, or the point set R2 of the coordinates not falling within the reasonable nipple position interval;

[0086] Screen the coordinates of multiple nipple positions to obtain the nipple position. Specifically:

[0087] Since most nipple positions will be concentrated within a reasonable nipple position interval, according to this reasonable nipple position interval, the embodiments of the present invention screen the nipple position coordinates output by the detection model. Among them, the output result may be the point set R1 of the coordinates falling within the reasonable nipple position interval, or the point set R2 of the points not falling within the reasonable nipple position. If R1 is not empty, select the coordinate corresponding to the maximum confidence level in the R1 point set as the nipple position. If R1 is empty, select the coordinate corresponding to the maximum confidence level in R2 as the nipple position.

[0088] Preferably, during the nipple detection process in the embodiments of the present invention, the operation interface of the detection workstation can be used to display the coronal plane image and the nipple position. When the nipple position is moved, the nipple position can be manually modified.

[0089] In the embodiments of the present invention, different from the mammography image, the image obtained by three-dimensional breast ultrasound is a three-dimensional image, and the three-dimensional breast ultrasound image also has information of different depths. During the ultrasound scanning process, as the ultrasound probe moves, the nipple position will also change. In the nipple detection method of the embodiments of the present invention, according to the nipple AI detection of the three-dimensional breast ultrasound image, when using three-dimensional breast ultrasound, manual confirmation of the nipple position is no longer required during the scanning process, solving the pain point that manual confirmation of the nipple position is required during the current three-dimensional breast ultrasound examination; saving manpower, being objective and efficient, and improving the scanning efficiency.

[0090] In addition, in the nipple detection of the embodiments of the present invention, the image of the three-dimensional ultrasound data is processed. Different from the traditional image processing method, a detection model is obtained by using a deep learning algorithm. The detection model is obtained through training with a large amount of data and has a higher detection accuracy compared with the traditional algorithm. Practice has proved that the detection rate in a total of 733 dcm is 99.45%. Among them, the detection rate of 699 M22 machine-scanned images is 99.71%, and the detection rate of 34 M60 machine-scanned images is 96.9697%. At the same time, it takes a short time. On average, it takes 0.587 s (python) from preprocessing to outputting the detection result for each dcm. Among them, the average time for detection + screening is 0.017 s. That is, the embodiments of the present invention can efficiently find the nipple position in the three-dimensional breast ultrasound scan image.

[0091] As Figure 5 shown, it shows a block diagram of a nipple detection device provided by an embodiment of the present application. The device has the functions of implementing the above method embodiments. The functions can be implemented by hardware or by hardware executing corresponding software. The device can be a computer device or can be set on a computer device.

[0092] As Figure 5 shown, a nipple detection device provided by an embodiment of the present invention, the device includes:

[0093] An image acquisition module, configured to acquire a breast ultrasound image file, automatically perform alignment at the middle position of the coronal plane, and acquire a coronal plane image of the AP position;

[0094] An image preprocessing module, configured to perform preprocessing of image data on the coronal plane image;

[0095] A detection module, which is used to input the pre-processed image into a detection model to obtain the reasonable nipple positions and the confidence levels of each reasonable position. The detection model is obtained by training the yolov5 network based on the training set of the coronal plane images;

[0096] A nipple position acquisition module, which is used to obtain the nipple position according to the reasonable nipple positions and the confidence levels.

[0097] As Figure 6 shown, the image pre-processing module specifically includes:

[0098] A selection unit, which is used to select the images in a predetermined depth interval of the coronal plane image located at the topmost layer;

[0099] A screenshot unit, which is used to intercept five pictures each in three specified depth intervals in the images in the predetermined depth interval;

[0100] An image calculation unit, which is used to perform a unified image pixel point spacing (spacing) operation on the five pictures obtained in each of the three depth intervals, and add and average the pixel values corresponding to the five pictures in the same depth interval to obtain one coronal plane picture for each of the three depth intervals. These three coronal plane pictures are respectively placed into the three RGB channels.

[0101] As Figure 6 shown, in the three-dimensional ultrasonic image of the embodiment of the present invention, the selection unit selects the images in a predetermined depth interval of the coronal plane image located at the topmost layer. Since obvious nipple information is usually located in the images in the depth interval of 1-4 mm from the topmost layer of the coronal plane image, the selection unit can preferentially select the images in this depth interval of the coronal plane image. However, at the same time, in order to retain the nipple position information at different depths as much as possible, as Figure 6 shown, the screenshot unit of the embodiment of the present invention performs the following operations on the images in this depth interval:

[0102] First, intercept five pictures each at three specified depths in this depth interval, that is, intercept five consecutive pictures at the first depth, for example, [1-2.2] mm, five consecutive pictures at the second depth, such as [2-3.2] mm, and five consecutive pictures at the third depth, such as [3-4.2] mm, a total of 15 pictures.

[0103] Next, the image calculation unit of the embodiment of the present invention respectively performs a unified image pixel point spacing (spacing) operation on the five pictures obtained in each of the three depth intervals, and adds and averages the pixel values corresponding to the five pictures in the same interval to generate one picture. At this time, one processed coronal plane picture is obtained for each of the three intervals. Then, these three pictures are respectively placed into the three RGB channels of the same picture.

[0104] Through the above steps of processing the coronal plane image by the image and processing module, a preprocessed image is obtained and input into the nipple detection model.

[0105] Preferably, the detection module specifically includes:

[0106] An output unit, configured to obtain the coordinates of multiple nipple positions output by the nipple detection model, and obtain a point set R1 of coordinates that fall within a reasonable nipple position interval, or a point set R2 of coordinates that do not fall within a reasonable nipple position interval;

[0107] A confidence unit, configured to obtain the confidence of the coordinates in each point set.

[0108] Preferably, the nipple position acquisition module is specifically configured to:

[0109] If R1 is not empty, select the coordinate with the maximum confidence in the point set R1 as the nipple position;

[0110] If R1 is empty, select the coordinate with the maximum confidence in R2 as the nipple position.

[0111] It should be noted that when the device provided in the above embodiment realizes its functions, only the above division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0112] In addition, the device provided in the above embodiment and the method embodiment belong to the same concept. For the specific implementation process, please refer to the method embodiment and will not be elaborated here.

[0113] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the nipple detection method as described above.

[0114] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the nipple detection method as described above.

[0115] Please refer to Figure 7 , which shows a schematic structural diagram of a computer device 1500 provided by an embodiment of the present application. The computer device 1500 can be used to implement the nipple detection method in the image provided in the above embodiment.

[0116] Specifically:

[0117] The computer device 1500 includes a central processing unit (CPU) 1501, a system memory 1504 including a random access memory (RAM) 1502 and a read-only memory (ROM) 1503, and a system bus 1505 connecting the system memory 1504 and the central processing unit 1501. The computer device 1500 also includes a basic input / output system (I / O system) 1506 for facilitating information transfer between various components within the computer, and a mass storage device 1507 for storing an operating system 1513, application programs 1514, and other program modules 1515.

[0118] The basic input / output system 1506 includes a display 1508 for displaying information and input devices 1509 such as a mouse and keyboard for user input of information. Among them, both the display 1508 and the input devices 1509 are connected to the central processing unit 1501 through an input / output controller 1510 connected to the system bus 1505. The basic input / output system 1506 may also include an input / output controller 1510 for receiving and processing inputs from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1510 also provides output to a display screen, printer, or other types of output devices.

[0119] The mass storage device 1507 is connected to the central processing unit 1501 through a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1507 and its associated computer-readable medium provide non-volatile storage for the computer device 1500. That is to say, the mass storage device 1507 may include computer-readable media (not shown) such as a hard disk or a CD-ROM drive.

[0120] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, magnetic tape cartridges, tapes, disk storage or other magnetic storage devices.

[0121] Of course, those skilled in the art will understand that the computer storage media is not limited to the above several types. The above-mentioned system memory 1504 and mass storage device 1507 may be collectively referred to as memory.

[0122] According to various embodiments of the present application, the computer device 1500 may also run on a remote computer on the network through a network such as the Internet. That is, the computer device 1500 may be connected to the network 1512 through the network interface unit 1511 connected to the system bus 1505. Or rather, the network interface unit 1511 may also be used to connect to other types of networks or remote computer systems (not shown).

[0123] The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by one or more processors. The above one or more programs include those for implementing the nipple detection method in the above image.

[0124] In an exemplary embodiment, a computer device is further provided. The computer device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory. The at least one instruction, at least one program, the code set or the instruction set is configured to be executed by the processor to implement the nipple detection method in the above image.

[0125] In an exemplary embodiment, a computer-readable storage medium is further provided. At least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set implements the nipple detection method in the above image when executed by a processor of a terminal. Optionally, the above computer-readable storage medium may be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0126] In an exemplary embodiment, a computer program product is further provided. When the computer program product is executed, it is used to implement the nipple detection method in the above image.

[0127] It should be understood that "a plurality of" mentioned herein refers to two or more. "And / or"

[0128] describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0129] In addition, the step numbers described in this document only exemplarily show one possible execution sequence among the steps. In some other embodiments, the above steps may not be executed in the numbered order. For example, two steps with different numbers may be executed simultaneously, or two steps with different numbers may be executed in the reverse order of the illustration. The embodiments of the present application do not limit this.

[0130] The above are only exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

[0131] The above are only partial implementation manners of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements can be made without departing from the principle of the present invention, and these improvements and refinements should also be regarded as within the protection scope of the present invention.

Claims

1. A nipple detection method, characterized in that, The method includes: Obtain a breast ultrasound image file, automatically align the middle position of the coronal plane, and obtain a coronal plane image in the AP position; Perform preprocessing on the image data of the coronal plane image. The specific steps for performing preprocessing on the image data of the coronal plane image include: Select an image in a predetermined depth interval located at a distance from the topmost coronal plane image; In the image in the predetermined depth interval, intercept five pictures each from three specified depth intervals; Perform a unified operation on the image pixel point spacing (spacing) for each of the five pictures obtained in the three depth intervals; Add the pixel values of the five pictures corresponding to the same depth interval and then take the average to obtain one coronal plane picture for each of the three depth intervals; Put these three coronal plane pictures into the RGB three channels respectively; Input the preprocessed image into a detection model to obtain the reasonable nipple position and the confidence level of each reasonable position. The detection model is obtained by training the yolov5 network based on the training set of the coronal plane image; Obtain the nipple position according to the reasonable nipple position and the confidence level.

2. The nipple detection method according to claim 1, wherein The step of inputting the preprocessed image into the nipple detection model to obtain the reasonable nipple position and the confidence level of each reasonable position specifically includes: The nipple detection model outputs the coordinates of multiple nipple positions, obtaining a point set R1 of the coordinates that fall within the reasonable nipple position interval, or a point set R2 of the coordinates that do not fall within the reasonable nipple position interval; At the same time, obtain the confidence levels of the coordinates in each point set.

3. The nipple detection method according to claim 2, characterized in that, The step of obtaining the nipple position according to the reasonable nipple position and the confidence level specifically includes: If R1 is not empty, select the coordinate with the highest confidence level in the point set R1 as the nipple position; If R1 is empty, select the coordinate with the highest confidence level in R2 as the nipple position.

4. A nipple detection device, characterized in that, The device includes: An image acquisition module, used to obtain a breast ultrasound image file, automatically align the middle position of the coronal plane, and obtain a coronal plane image in the AP position; An image preprocessing module, used to perform preprocessing on the image data of the coronal plane image. The image preprocessing module specifically includes: A selection unit, used to select an image in a predetermined depth interval located at a distance from the topmost coronal plane image; A screenshot unit, used to intercept five pictures each from three specified depth intervals in the image in the predetermined depth interval; An image calculation unit, used to perform a unified operation on the image pixel point spacing (spacing) for each of the five pictures obtained in the three depth intervals, add the pixel values of the five pictures corresponding to the same depth interval and then take the average to obtain one coronal plane picture for each of the three depth intervals, and put these three coronal plane pictures into the RGB three channels respectively; A detection module, used to input the preprocessed image into a detection model to obtain the reasonable nipple position and the confidence level of each reasonable position. The detection model is obtained by training the yolov5 network based on the training set of the coronal plane image; A nipple position acquisition module, used to obtain the nipple position according to the reasonable nipple position and the confidence level.

5. The nipple detection device according to claim 4, characterized in that, The detection module specifically includes: An output unit, configured to obtain the coordinates of multiple nipple positions output by the nipple detection model, to obtain a point set R1 of coordinates that fall within a reasonable nipple position interval, or a point set R2 of coordinates that do not fall within a reasonable nipple position interval; A confidence unit, configured to obtain the confidence of the coordinates in each point set.

6. The nipple detection device according to claim 5, characterized in that The nipple position obtaining module is specifically configured to: If R1 is not empty, select the coordinate with the maximum confidence in the point set R1 as the nipple position; If R1 is empty, select the coordinate with the maximum confidence in R2 as the nipple position.

7. A computer device, characterized in that, The computer device includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the nipple detection method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that, At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the nipple detection method according to any one of claims 1 to 3.

Images