Nodule orientation identification device, method, storage medium and electronic device

Abstract

The application provides a nodule orientation identification device, method, storage medium and electronic equipment. The nodule orientation identification device comprises: a medical image acquisition module configured to acquire a medical image, the medical image comprising a nodule and a reference object; a nodule orientation identification module configured to process the medical image to obtain coordinate information of the nodule and coordinate information of the reference object, and identify the orientation of the nodule according to the coordinate information of the nodule and the coordinate information of the reference object. The nodule orientation identification device can automatically realize nodule orientation detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to a nodule recognition device, in particular to a nodule orientation recognition device, method, storage medium and electronic equipment. BACKGROUND

[0002] Nodule orientation detection is an important research direction in the field of medical image analysis, and its main purpose is to determine the orientation of the nodule according to the medical image. Through nodule orientation detection, the doctor's diagnosis efficiency and accuracy can be greatly improved, the doctor's work burden can be reduced, and more timely and accurate treatment can be provided for the patient. However, in actual application, medical images have complex structures and rich information, and the shape, size, position and other differences of the nodule in the image are large. Therefore, how to quickly and accurately realize nodule orientation detection has become one of the technical problems to be solved by the technical personnel in the related field. SUMMARY

[0003] The application provides a nodule orientation recognition device for quickly and accurately realizing nodule orientation detection.

[0004] In a first aspect, an embodiment of the application provides a nodule orientation recognition device, which comprises: a medical image acquisition module configured to acquire a medical image, the medical image comprising a nodule and a reference object; and a nodule orientation recognition module configured to process the medical image to obtain coordinate information of the nodule and coordinate information of the reference object, and recognize the orientation of the nodule according to the coordinate information of the nodule and the coordinate information of the reference object.

[0005] In an implementation form of the first aspect, the nodule orientation recognition module comprises a coordinate information acquisition unit configured to process the medical image by using a detection and recognition model to obtain the coordinate information of the nodule and the coordinate information of the reference object, the detection and recognition model being a trained yolov8 multi-task model.

[0006] In an implementation form of the first aspect, the nodule orientation recognition module further comprises: a first recognition unit configured to process the coordinate information of the nodule and the coordinate information of the reference object by using the detection and recognition model to obtain a first recognition result of the nodule orientation; a second recognition unit configured to process the coordinate information of the nodule and the coordinate information of the reference object by using a coordinate classification model to obtain a second recognition result of the nodule orientation, the coordinate classification model being a trained SVM machine learning model; and a result fusion unit configured to fuse the first recognition result and the second recognition result to obtain the orientation of the nodule.

[0007] In an implementation form of the first aspect, the obtaining method of the detection and recognition model comprises: constructing a yolov8 initial model, performing attention fusion on the output of a DFL layer of the yolov8 initial model, and adding a nodule direction classification branch to the yolov8 initial model.

[0008] In an implementation form of the first aspect, the training method of the detection and recognition model comprises: step a, obtaining first training data, the first training data comprising first training images and coordinate annotation results and nodule direction annotation results of the first training images, the first training images comprising a first type of image, a second type of image, a third type of image, and a fourth type of image, the first type of image not containing the nodule and the reference object, the second type of image containing the nodule and not containing the reference object, the third type of image containing the reference object and not containing the nodule, and the fourth type of image containing the nodule and the reference object; step b, inputting the first training images into the detection and recognition model to obtain output data of a DFL layer of the detection and recognition model as first data; step c, inputting the first data into a focus layer to extract features, and connecting the feature extraction results to obtain second data; step d, inputting the second data into an attention layer to obtain third data; step e, inputting the third data into a softmax layer to obtain feature weights; step f, weighting the second data using the feature weights, and performing pixel-by-pixel addition in the first dimension on the weighted second data to obtain fourth data; step g, inputting the fourth data into a classification head to obtain output results; step h, obtaining a loss function according to the output results and the coordinate annotation results and the nodule direction annotation results of the first training images; and step i, adjusting parameters of the detection and recognition model according to the loss function, and repeating steps b to i until a training termination condition is met.

[0009] In an implementation form of the first aspect, the training method of the coordinate classification model comprises: obtaining second training data, the second training data comprising second training images and nodule direction annotation results of the second training images; processing the second training images using the detection and recognition model to obtain predicted coordinates of the nodule and predicted coordinates of the reference object; and training the coordinate classification model using the predicted coordinates of the nodule and the predicted coordinates of the reference object and the nodule direction annotation results of the second training images.

[0010] In an implementation form of the first aspect, the nodule is a thyroid nodule, and the reference object is a trachea.

[0011] In a second aspect, an embodiment of the present application provides a nodule direction identification method, the nodule direction identification method comprising: obtaining a medical image, the medical image comprising a nodule and a reference object; processing the medical image to obtain coordinate information of the nodule and coordinate information of the reference object, and identifying a direction of the nodule according to the coordinate information of the nodule and the coordinate information of the reference object.

[0012] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the nodule direction identification method in the second aspect.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises: a memory storing a computer program; and a processor connected to the memory in communication, and the processor executes the nodule direction identification method in the second aspect when the computer program is invoked.

[0014] The nodule direction identification device provided by the embodiment of the present application can automatically realize nodule direction detection, and the process basically does not need human intervention, and the efficiency is high. In addition, the nodule direction identification device identifies the direction of the nodule according to the coordinate information of the reference object and the coordinate information of the nodule, and the accuracy of the identification result is high. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 An application scenario schematic diagram of the nodule direction identification device in an embodiment of the present application is shown.

[0016] Figure 2 A structure schematic diagram of the nodule direction identification device in an embodiment of the present application is shown.

[0017] Figure 3 A structure schematic diagram of the nodule direction identification module in an embodiment of the present application is shown.

[0018] Figure 4 A structure schematic diagram of the attention layer in an embodiment of the present application is shown.

[0019] Figure 5 A training process flowchart of the coordinate classification model in an embodiment of the present application is shown.

[0020] Figure 6 A flowchart of the nodule direction identification method in an embodiment of the present application is shown.

[0021] Figure 7 A structure schematic diagram of the electronic device in an embodiment of the present application is shown.

[0022] ELEMENT NUMBER EXPLANATION

[0023] 1 electronic device

[0024] 11 medical image acquisition device

[0025] 12 general-purpose processor

[0026] 121 central processing unit

[0027] 122 neural network processing unit

[0028] 13 display

[0029] 14 memory

[0030] 2 nodule orientation identification device

[0031] 21 medical image acquisition module

[0032] 22 nodule orientation identification module

[0033] 221 coordinate information acquisition unit

[0034] 222 first identification unit

[0035] 223 second identification unit

[0036] 224 result fusion unit

[0037] 7 electronic device

[0038] 71 memory

[0039] 72 processor

[0040] 73 display

[0041] S51-S53 steps

[0042] S61-S62 steps DETAILED DESCRIPTION

[0043] The present application will be described by specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied by other different specific embodiments, and various modifications or changes can be made to the details in the present specification based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0044] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0045] In practical applications, medical images possess complex structures and rich information, and nodules vary considerably in morphology, size, and location within these images. Therefore, how to quickly and accurately detect the location of nodules has become one of the urgent technical problems that professionals in related fields need to solve.

[0046] To address at least the aforementioned issues, this application provides a nodule orientation identification device. This device can automatically detect nodule orientation with minimal human intervention, resulting in high efficiency. Furthermore, the device identifies the nodule's orientation based on the coordinate information of a reference object and the nodule's coordinate information, achieving high accuracy.

[0047] The nodule identification device provided in this application embodiment can be applied to electronic devices, including but not limited to desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other terminal devices. It also includes applications in databases, servers, and service response systems based on terminal artificial intelligence. This application embodiment does not impose any restrictions on the specific type of electronic device.

[0048] Figure 1 The diagram shown is a structural schematic of an electronic device 1 in one embodiment of this application. The nodule orientation identification device provided in this embodiment can be applied to... Figure 1 The electronic device 1 shown is an example, but not limited to it. For example... Figure 1 As shown, the electronic device 1 includes a medical image acquisition device 11, at least one general-purpose processor 12, a display 13, and a memory 14.

[0049] The medical image acquisition device 11 can be any device capable of acquiring medical images. In the embodiments of this application, the medical image acquisition device 11 is communicatively connected to a medical image acquisition device and is used to acquire medical images from the medical image acquisition device. The medical image acquisition device may include, for example, an ultrasound scanning device, a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, etc.

[0050] In some possible implementation manners, the medical image acquisition apparatus 11 can also include a scanning device capable of directly scanning a patient to acquire a medical image.

[0051] The general processor 12 can be any type of apparatus capable of processing electronic instructions, and the electronic device 1 can include one or more general processors 12 in the embodiments of the present application, for example, one or both of a central processing unit (CPU) 121 and a neural-network processing unit (NPU) 122. In addition, one or more of a graphics processing unit (GPU), a microprocessor, a microcontroller, a host processor, a controller, and an application specific integrated circuit (ASIC) can also be included. The general processor 12 is configured to execute various types of digital storage instructions, such as software or firmware programs stored in the memory 14, which can enable the electronic device 1 to provide a variety of services. For example, the processor 11 can execute programs or process data to implement at least part of the function of the nodule orientation identification apparatus provided in the embodiments of the present application.

[0052] The function of the central processing unit 121 is mainly to analyze computer instructions and process data in computer software, to realize the control of the whole electronic device 1, and to control and allocate all hardware resources (such as storage resources, communication resources, I / O interfaces, etc.) of the electronic device 1.

[0053] The neural-network processing unit is a new type of processor based on neural-network algorithms and acceleration, which is specially designed for artificial intelligence and used to accelerate the operation of neural networks to solve the problem of low efficiency of traditional chips in neural-network operation.

[0054] It should be noted that the name of the neural-network processing unit does not constitute a limitation on the present application, for example, in other application scenarios, the neural-network processing unit can also be deformed or replaced by other similar functional processors, such as a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0055] The display 13 can specifically include a display screen (display panel). In some implementations, the display panel can be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. The display device can also be a touch panel (touch screen, touch screen), which can include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation thereon or adjacent thereto, the central processor 121 is transmitted to determine the type of touch event, and then the central processor 121 provides a corresponding visual output on the display device according to the type of touch event.

[0056] The memory 14 can include a volatile memory (Volatile Memory), such as a Random Access Memory (RAM), a cache. The memory 14 can also include a non-volatile memory (Non-Volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory, a Hard Disk Drive (HDD), or a Solid-State Drive (SSD). The memory 14 can be used to store data such as images collected by the image collection device 11. The memory 14 can also be used to store program instructions for the processor to call and implement the functions of the nodule orientation identification device provided by the embodiments of the present application.

[0057] Figure 2 The display is a structural schematic diagram of the nodule orientation identification device 2 in the embodiments of the present application. As shown in the figure, the nodule orientation identification device 2 includes a medical image acquisition module 21 and a nodule orientation identification module 22. Figure 2

[0058] The medical image acquisition module 21 is configured to acquire a medical image, which includes a nodule and a reference object. The reference object can be a structure such as tissue, organs, etc. near the nodule.

[0059] In some possible implementations, the medical image is an ultrasound image. The ultrasound image refers to the use of high-frequency sound waves to create an image of the nodule. According to the ultrasound image, cystic lesions, solid lesions, nodules, lymph nodes, etc. can be detected, so as to evaluate the size, shape, structure and blood flow of the nodule.

[0060] ​In some possible implementations, the medical image can be a CT image. The CT image is a medical image generated using X-rays and computers, which can provide high-resolution and high-sensitivity nodule images and can be used to evaluate the shape, size, density, and location of the nodule. According to the CT image, a nodule smaller than 1 millimeter can be detected, and the nodule and the surrounding tissue can be better distinguished.

[0061] In some possible implementations, the medical image can be an MRI image. The MRI image is a three-dimensional image of the human body inside generated using a magnetic field and radio waves, which can be used to evaluate the morphology, histology, blood supply, and enhancement of the nodule.

[0062] The nodule orientation identification module 22 is connected to the medical image acquisition module 21 and is configured to process the medical image to obtain coordinate information of the nodule and coordinate information of the reference object, and identify the orientation of the nodule according to the coordinate information of the nodule and the coordinate information of the reference object. The coordinate information of the nodule is used to determine the position of the nodule, for example, which can be represented in the form of a rectangular box. The coordinate information of the reference object is used to determine the position of the reference object. The orientation of the nodule can be, for example, the left lobe, the right lobe, or the isthmus.

[0063] According to the above description, the nodule orientation identification device 2 provided by the embodiments of the present application can automatically realize nodule orientation detection, which basically does not require human intervention and has high efficiency. In addition, the nodule orientation identification device 2 identifies the orientation of the nodule according to the coordinate information of the reference object and the coordinate information of the nodule, and the accuracy of the identification result is high.

[0064] Please refer to Figure 3 In an embodiment of the present application, the nodule orientation identification module 22 includes a coordinate information acquisition unit 221. The coordinate information acquisition unit 221 is configured to process the medical image by using a detection and identification model to obtain the coordinate information of the nodule and the coordinate information of the reference object. The detection and identification model is a trained yolov8 multi-task model. Yolov8 is a target detection algorithm based on deep learning, which regards target detection as a regression problem, inputs the entire medical image into a neural network, and directly outputs the position and category information of the nodule and the reference object, so as to quickly and accurately obtain the coordinate information of the nodule and the coordinate information of the reference object.

[0065] In an embodiment of the present application, the nodule orientation identification module 22 can further include a first identification unit 222, a second identification unit 223, and a result fusion unit 224.

[0066] The first identification unit 222 is configured to process the coordinate information of the nodule and the coordinate information of the reference object by using a detection identification model to obtain a first identification result of the nodule position. The first identification result is a classification probability of the nodule position obtained by the detection identification model. For example, the first identification result can be a probability p_11 that the nodule is located in the left lobe, a probability p_12 that the nodule is located in the right lobe, and a probability p_13 that the nodule is located in the isthmus.

[0067] The second identification unit 223 is configured to process the coordinate information of the nodule and the coordinate information of the reference object by using a coordinate classification model to obtain a second identification result of the nodule position. The coordinate classification model is a trained support vector machine (SVM) model. The second identification result is a classification probability of the nodule position obtained by the coordinate classification model. For example, the second identification result can be a probability p_21 that the nodule is located in the left lobe, a probability p_22 that the nodule is located in the right lobe, and a probability p_23 that the nodule is located in the isthmus. The SVM model is a supervised learning machine learning model used for classification analysis of the nodule position. In the embodiments of the present application, the SVM model maps the data such as the coordinate information of the nodule and the coordinate information of the reference object into a high-dimensional space, and separates the data of different categories, so as to realize classification to obtain the second identification result. Specifically, in the SVM model, each data point is regarded as a point in an n-dimensional vector space, and the SVM model obtains the classification probability of the nodule position, i.e., the second identification result, by finding an optimal hyperplane (i.e., an optimal boundary separating the data). n is a positive integer, and the selection of the hyperplane is based on the maximization of the distance (i.e., the interval) between the data points and the hyperplane.

[0068] The result fusion unit 224 is configured to fuse the first identification result and the second identification result to obtain the position of the nodule. Specifically, the first identification result corresponds to one probability of each nodule position, and the second identification result corresponds to another probability of each nodule position. The nodule fusion unit 224 is used to fuse the two probabilities, and uses softmax to obtain the class with the maximum probability based on the fused probability, so as to obtain the final identification result of the nodule position.

[0069] In some possible implementation manners, the result fusion unit 224 fuses the first identification result and the second identification result by using an average value, but the present application is not limited thereto.

[0070] In an embodiment of the present application, the detection identification model is obtained by constructing a yolov8 initial model, performing attention fusion on the output of the DFL layer of the yolov8 initial model, and adding a nodule position classification branch to the yolov8 initial model.

[0071] In an embodiment of the present application, the training method of the detection and recognition model comprises steps a to i.

[0072] Step a, obtaining first training data. The first training data comprises first training images and their coordinate annotation results and nodule orientation annotation results, wherein the coordinate annotation results constitute a nodule and / or reference object detection dataset, and the nodule orientation annotation results constitute a nodule orientation classification dataset. The first training images comprise first type images, second type images, third type images and fourth type images. The first type images are pure shadow images and do not contain nodules and reference objects. The second type images contain nodules but do not contain reference objects. The third type images contain reference objects but do not contain nodules. The fourth type images contain nodules and reference objects.

[0073] In some possible implementations, the coordinate annotation results of the first training images can be obtained by annotating the positions of the nodules or reference objects with rectangular boxes, but the present application is not limited thereto.

[0074] Step b, inputting the first training images into the detection and recognition model and obtaining the output data of the DFL layer of the detection and recognition model as first data. For example, if the size of the input first training images is [640, 640, 3] and the training batchsize is N, the first data can comprise y1=[N, 66, 80, 80], y2=[N, 66, 40, 40] and y3=[N, 66, 20, 20].

[0075] Step c, inputting the first data into a focus layer for feature extraction and connecting the feature extraction results to obtain second data. Specifically, all the first data can be inputted into the focus layer for feature extraction, or part of the first data can be inputted into the focus layer for feature extraction. For example, if the output size of the focus layer is [MxN, 66, 20, 20], the size of the features obtained by inputting y1 into the focus layer is [16xN, 66, 20, 20], the size of the features obtained by inputting y2 into the focus layer is [4xN, 66, 20, 20], and y3 does not pass through the focus layer so that the size of its features is still [N, 66, 20, 20]. After concatenating the features of y1, the features of y2 and y3, the second data with a size of [21xN, 66, 20, 20] is obtained.

[0076] Step d, inputting the second data into an attention layer to obtain third data. Figure 4 An attention layer is shown as a structure of an embodiment of the present application, which can be used to process the second data in the embodiment of the present application. For example, after attention feature fusion is performed on the second data X with a size of [21xN, 66, 20, 20], the features Y with a dimension of [21xN, 1] can be obtained as the third data.

[0077] Step e, inputting the third data into a softmax layer to obtain feature weights W. The size of the feature weights W is, for example, [21xN].

[0078] Step f, weighting the second data using the feature weights and pixel-wise adding the weighted second data in the first dimension to obtain fourth data. Specifically, weighting the second data X using the feature weights W obtains WxX, and pixel-wise adding the weighted data in the first dimension can process the second data X with the size of [21xN, 66, 20, 20] into the fourth data with the size of [N, 66, 20, 20].

[0079] Step g, inputting the fourth data into a classification head to obtain an output result. In some implementations, the classification head can be composed of a fully connected layer (FC). For example, the classification head can scale the feature dimension 66x20x20 of the fourth data to the feature dimension 2.

[0080] Step h, obtaining a loss function according to the output result, the coordinate annotation result and the nodule orientation annotation result of the first training image.

[0081] In some possible implementations, the loss function L can be represented by the following formula 1, but the present application is not limited thereto.

[0082] L = w1xL CIOU + w2xL DFL + w3xL VFL1 + w4xL VFL2 , formula 1;

[0083] wherein, L CIOU , L DFL and L VFL1 are loss functions of yolov8, which can be obtained by prior art, L CIOU is a CIOU loss (Complete Intersection over Union Loss), L DFL is a DFL loss (Distribution FocalLoss), L VFL1 is a VFL loss (VariFocal loss). L VFL2(Asymmetrically weighted) is the nodule orientation classification loss, for example, in some embodiments can be four classification (left lobe, right lobe, isthmus and unable to determine) nodule orientation classification loss. w1, w2 and w3 are the default weight values of yolov8, which can be obtained by prior art. In addition, the weight value w4 in the embodiment of the application is configured as w4 = w1 + w2 + w3, so as to balance the detection and classification loss of the model.

[0084] Step i, adjusting the parameters of the detection recognition model according to the loss function, and repeating steps b to i until the training termination condition is met. The training termination condition can be configured according to actual needs.

[0085] Please refer to Figure 5 In an embodiment of the application, the training method of the coordinate classification model comprises the following steps S51 to S53.

[0086] S51, obtaining second training data, the second training data comprising second training images and their nodule orientation annotation results. Wherein, the second training images can be the same as the first training images, or can be different from the first training images.

[0087] S52, processing the second training images using the detection recognition model to obtain the predicted coordinates of the nodule and the predicted coordinates of the reference object.

[0088] S53, training the coordinate classification model using the predicted coordinates of the nodule and the predicted coordinates of the reference object, and the nodule orientation annotation results of the second training images.

[0089] In some possible implementations, the predicted coordinates of the nodule and the predicted coordinates of the reference object can be normalized to obtain normalized coordinates [x1, y1, cw, ch], wherein x1, y1, cw, ch are all floating-point numbers between 0 and 1. In particular, if the second training image does not contain the nodule or the reference object, x1, y1, cw, ch of the second training image are all configured as 0. Based on this, in step S53, the normalized nodule coordinates and reference object coordinates can be spliced into an array with a dimension of [1, 8] as x, and the true label of the second training image as y, and [x, y] pairs are input into the coordinate classification model for training.

[0090] In an embodiment of the application, the nodule is a thyroid nodule, and the reference object is a trachea.

[0091] In one embodiment of this application, for a target image, the nodule orientation recognition module processes it using a detection and recognition model to obtain nodule coordinates (box1) and trachea coordinates (box2), and obtains the nodule orientation classification probability. Subsequently, the nodule orientation recognition module normalizes box1 and box2 to obtain normalized coordinates, and processes the normalized coordinates using a coordinate classification model to obtain the nodule orientation classification probability. Based on this, the nodule orientation recognition module fuses the nodule orientation classification probabilities obtained from the detection and recognition model and the coordinate classification model to obtain the final nodule orientation recognition result.

[0092] This application also provides a method for nodule location identification. Figure 6 The flowchart shown is a process for nodule location identification in an embodiment of this application. Figure 6 As shown, the nodule orientation identification method provided in this application embodiment includes the following steps S61 and S62.

[0093] S61, Acquire medical images, including nodules and reference objects.

[0094] S62, Process the medical image to obtain the coordinate information of the nodule and the coordinate information of the reference object, and identify the orientation of the nodule based on the coordinate information of the nodule and the coordinate information of the reference object.

[0095] It should be noted that steps S61 and S62 above are the same as... Figure 2 The medical image acquisition module 21 and the nodule orientation recognition module 22 in the medical image acquisition device 2 shown are respectively corresponding, and will not be described in detail here.

[0096] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the nodule orientation identification method described in this application.

[0097] In the embodiments of this application, any combination of one or more storage media can be used. The storage medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0098] The application also provides an electronic device. Figure 7 Fig. 7 shows a schematic diagram of an electronic device according to an embodiment of the application. Figure 7 As shown in Fig. 7, the electronic device 7 according to the embodiment of the application comprises a memory 71 and a processor 72.

[0099] The memory 71 is configured to store computer programs; preferably, the memory 71 comprises a ROM, a RAM, a magnetic disc, a U disc, a memory card, an optical disc or other media capable of storing program codes.

[0100] Specifically, the memory 71 can comprise a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 7 can further comprise other removable / non-removable, volatile / non-volatile computer system storage media. The memory 71 can comprise at least one program product having a set (e.g. at least one) program modules configured to perform the functions of the embodiments of the application.

[0101] The processor 72 is connected to the memory 70 and is configured to execute the computer programs stored in the memory 71, so that the electronic device 7 performs the nodule orientation identification method.

[0102] Optionally, the processor 72 can be a general processor, including a central processing unit (CPU), a network processor (NP) and the like; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0103] Optionally, the electronic device 7 according to the embodiment of the application can further comprise a display 73. The display 73 is connected to the memory 71 and the processor 72 in communication and is configured to display the related GUI interface of the nodule orientation identification method.

[0104] In summary, the nodule direction identification device provided by the embodiments of the present application can automatically realize nodule direction detection, and the process basically does not require manual participation and has high efficiency. In addition, the nodule direction identification device identifies the direction of the nodule according to the coordinate information of the reference object and the coordinate information of the nodule, and the identification result has high accuracy. Therefore, the present application effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0105] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought disclosed by the present application should be covered by the claims of the present application.

Claims

1. A nodule location identification device, characterized in that, The nodule location identification device includes: A medical image acquisition module is configured to acquire medical images, including nodules and reference objects; The nodule orientation recognition module is configured to process the medical image to obtain the coordinate information of the nodule and the coordinate information of the reference object, and to identify the orientation of the nodule based on the coordinate information of the nodule and the coordinate information of the reference object; The nodule orientation identification module includes: The coordinate information acquisition unit is configured to process the medical image using a detection and recognition model to obtain the coordinate information of the nodule and the coordinate information of the reference object. The detection and recognition model is a trained YOLOv8 multi-task model. The method for obtaining the detection and recognition model is as follows: constructing an initial YOLOv8 model, performing attention fusion on the output of the DFL layer of the initial YOLOv8 model, and adding a nodule orientation classification branch to the initial YOLOv8 model. The first identification unit is configured to process the coordinate information of the nodule and the coordinate information of the reference object using the detection and identification model to obtain a first identification result of the nodule orientation; The second identification unit is configured to process the coordinate information of the nodule and the coordinate information of the reference object using a coordinate classification model to obtain a second identification result of the nodule orientation. The coordinate classification model is a trained SVM machine learning model. The result fusion unit is configured to fuse the first identification result and the second identification result to obtain the orientation of the nodule.

2. The nodule orientation identification device according to claim 1, characterized in that, The training method for the detection and recognition model includes: Step a, obtain the first training data, which includes the first training image and its coordinate annotation results and nodule orientation annotation results. The first training image includes a first type of image, a second type of image, a third type of image and a fourth type of image. The first type of image does not contain the nodule and the reference object. The second type of image contains the nodule but does not contain the reference object. The third type of image contains the reference object but does not contain the nodule. The fourth type of image contains the nodule and the reference object. Step b: Input the first training image into the detection and recognition model, and obtain the output data of the DFL layer of the detection and recognition model as the first data; Step c: Input the first data into the focusing layer for feature extraction, and then concatenate the feature extraction results to obtain the second data; Step d: Input the second data into the attention layer to obtain the third data; Step e: Input the third data into the softmax layer to obtain feature weights; Step f: The second data is weighted using the feature weights, and the weighted second data is summed pixel by pixel in the first dimension to obtain the fourth data; Step g: Input the fourth data into the classification head to obtain the output result; Step h: Obtain the loss function based on the output result, the coordinate annotation result of the first training image, and the nodule orientation annotation result; Step i: Adjust the parameters of the detection and recognition model according to the loss function, and repeat steps b to i until the training termination condition is met.

3. The nodule orientation identification device according to claim 1, characterized in that, The training method for the coordinate classification model includes: Acquire second training data, which includes second training images and their nodule orientation annotation results; The detection and recognition model is used to process the second training image to obtain the predicted coordinates of the nodule and the predicted coordinates of the reference object; The coordinate classification model is trained using the predicted coordinates of the nodules, the predicted coordinates of the reference object, and the nodule orientation annotation results of the second training image.

4. The nodule orientation identification device according to claim 1, characterized in that, The nodule is a thyroid nodule, and the reference object is the trachea.

5. A method for nodule location identification, characterized in that, The nodule orientation identification device applied to any one of claims 1 to 4, wherein the nodule orientation identification method comprises: Acquire medical images, including nodules and reference objects; The medical image is processed to obtain the coordinate information of the nodule and the coordinate information of the reference object, and the orientation of the nodule is identified based on the coordinate information of the nodule and the coordinate information of the reference object.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the nodule orientation identification method as described in claim 5.

7. An electronic device, characterized in that, The electronic device includes: A memory that stores a computer program; The processor, which is communicatively connected to the memory, executes the nodule location identification method according to claim 5 when calling the computer program.

Images